Disentanglement

dSprites is a dataset of 2D shapes procedurally generated from 6 ground truth independent latent factors. These factors are color, shape, scale, rotation, x and y positions of a sprite. All possible combinations of these latents are present exactly once, generating N = 737280 total images. Source: dSprites

The smallNORB dataset is a datset for 3D object recognition from shape. It contains images of 50 toys belonging to 5 generic categories: four-legged animals, human figures, airplanes, trucks, and cars. The objects were imaged by two cameras under 6 lighting conditions, 9 elevations (30 to 70 degrees every 5 degrees), and 18 azimuths (0 to 340 every 20 degrees). The training set is composed of 5 instances of each category (instances 4, 6, 7, 8 and 9), and the test set of the remaining 5 instances (instances 0, 1, 2, 3, and 5). Source: https://cs.nyu.edu/~ylclab/data/norb-v1.0-small/ Image Source: https://www.kaggle.com/nepuerto/the-small-norb-dataset-v10

The Sprites dataset contains 60 pixel color images of animated characters (sprites). There are 672 sprites, 500 for training, 100 for testing and 72 for validation. Each sprite has 20 animations and 178 images, so the full dataset has 120K images in total. There are many changes in the appearance of the sprites, they differ in their body shape, gender, hair, armor, arm type, greaves, and weapon. Source: Challenges in Disentangling Independent Factors of Variation

3dshapes is a dataset of 3D shapes procedurally generated from 6 ground truth independent latent factors. These factors are floor colour, wall colour, object colour, scale, shape and orientation. Source: 3D Shapes Dataset

Car CAD models from "3d object detection and viewpoint estimation with a deformable 3d cuboid model" were used to generate the dataset. For each of the 199 car models, the authors generated $64\times64$ color renderings from 24 rotation angles each offset by 15 degrees, as well as from 4 different camera elevations.

Update on 3DIdent, where we introduce six additional object classes (Hare, Dragon, Cow, Armadillo, Horse, and Head), and impose a causal graph over the latent variables. For further details, see Appendix B in the associated paper (https://arxiv.org/abs/2106.04619).

A data-set which consists of over one million images of physical 3D objects with seven factors of variation, such as object color, shape, size and position. Source: On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

Novel benchmark which features aspects of natural scenes, e.g. a complex 3D object and different lighting conditions, while still providing access to the continuous ground-truth factors. We use the Blender rendering engine to create visually complex 3D images. Each image in the dataset shows a colored 3D object which is located and rotated above a colored ground in a 3D space. Additionally, each scene contains a colored spotlight which is focused on the object and located on a half-circle around the scene. The observations are encoded with an RGB color space, and the spatial resolution is 224x224 pixels. The images are rendered based on a 10-dimensional latent, where: (1) three dimensions describe the XYZ position, (2) three dimensions describe the rotation of the object in Euler angles, (3) two dimensions describe the color of the object and the ground of the scene, respectively, and (4) two dimensions describe the position and color of the spotlight. We use the HSV color space to describe the color of the object and the ground with only one latent each by having the latent factor control the hue value. The training set and test set contain 250,000 and 25,000 observation-latent pairs, respectively, whereby the latents are uniformly sampled from the unit hyperrectangle.

Largest, first-of-its-kind, in-the-wild, fine-grained workout/exercise posture analysis dataset, covering three different exercises: BackSquat, Barbell Row, and Overhead Press. Seven different types of exercise errors are covered. Unlabeled data is also provided to facilitate self-supervised learning.

This Dataset consists of 2120 sequences of binary masks of pedestrians. The sequence length varies between 2-710. For details, we refer to our paper. It is based on the original KITTI Segmentation challenge which can be found at https://www.vision.rwth-aachen.de/page/mots A detailed description can be found at: https://openreview.net/pdf?id=EbIDjBynYJ8 An example dataloader can be found at: https://github.com/bethgelab/slow_disentanglement/

This dataset derives from Coil100. There are more than 1,1M images of 100 objects. Each object was turned on a turnable through 360 degrees to vary object pose with respect to a fixed color camera. Images of the objects were taken at pose intervals of 5 degrees. This corresponds to 72 poses per object. Then planar rotation (9 angles) and 18 scaling factors has been applied. Objects have a wide variety of complex geometric and reflectance characteristics. This augmented version of Coil100 has been design for Disentangled Representation Learning for real images, the Factors of Variations are: | Factors | # values | |----------|----------| | Object | 100 | | 3D Pose | 72 | | Rotation | 9 | | Scale | 18 | The binarized version is also available.

This csv consists of (x-position, y-position, area) tuples of three views (left, middle, right) of downscaled binary masks with aspect ratio kept (64 x 128) from the 2019 YouTube-VIS challenge, which can be found at https://competitions.codalab.org/competitions/20127#participate-get-data. Extracting pairs from this csv results in 234,652 transitions in the given statistics. These statistics can be used to augment ground truth factor distributions with natural transitions, which we demonstrate with spriteworld. For details, we refer to our paper, which can be found at https://openreview.net/forum?id=EbIDjBynYJ8.

Synthetic dataset intended for benchmarking disentanglement frameworks. XYSquares is adversarial in nature; the distance between any two observations in the dataset is constant when measured using a pixel-wise distance function. It is usually impossible for VAE frameworks that use pixel-wise losses to disentangle this dataset. The dataset is constructed from 3 non-overlapping red, green and blue squares that are each $8\times8$ pixels in size. Each square can move along the $x$ and $y$ coordinates of an $8\times8$ grid. The resulting images are $64\times64$ pixels in size. With this construction the dataset has a total of 8 ground-truth factors for a total of $8^6 = 262144$ observations.