Object Recognition

The ImageNet dataset contains 14,197,122 annotated images according to the WordNet hierarchy. Since 2010 the dataset is used in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), a benchmark in image classification and object detection. The publicly released dataset contains a set of manually annotated training images. A set of test images is also released, with the manual annotations withheld. ILSVRC annotations fall into one of two categories: (1) image-level annotation of a binary label for the presence or absence of an object class in the image, e.g., “there are cars in this image” but “there are no tigers,” and (2) object-level annotation of a tight bounding box and class label around an object instance in the image, e.g., “there is a screwdriver centered at position (20,25) with width of 50 pixels and height of 30 pixels”. The ImageNet project does not own the copyright of the images, therefore only thumbnails and URLs of images are provided. * Total number of non-empty WordNet synsets: 21841 * Total number of images: 14197122 * Number of images with bounding box annotations: 1,034,908 * Number of synsets with SIFT features: 1000 * Number of images with SIFT features: 1.2 million Source: ImageNet Large Scale Visual Recognition Challenge Image Source: https://cs.stanford.edu/people/karpathy/cnnembed/

The ModelNet40 dataset contains synthetic object point clouds. As the most widely used benchmark for point cloud analysis, ModelNet40 is popular because of its various categories, clean shapes, well-constructed dataset, etc. The original ModelNet40 consists of 12,311 CAD-generated meshes in 40 categories (such as airplane, car, plant, lamp), of which 9,843 are used for training while the rest 2,468 are reserved for testing. The corresponding point cloud data points are uniformly sampled from the mesh surfaces, and then further preprocessed by moving to the origin and scaling into a unit sphere. Source: Geometric Feedback Network for Point Cloud Classification

CORe50 is a dataset designed for assessing Continual Learning techniques in an Object Recognition context. Source: CORe50: a New Dataset and Benchmark for Continuous Object Recognition

The 'shape bias' dataset was introduced in Geirhos et al. (ICLR 2019) and consists of 224x224 images with conflicting texture and shape information (e.g., cat shape with elephant texture). This is used to measure the shape vs. texture bias of image classifiers.

The Neuromorphic-Caltech101 (N-Caltech101) dataset is a spiking version of the original frame-based Caltech101 dataset. The original dataset contained both a "Faces" and "Faces Easy" class, with each consisting of different versions of the same images. The "Faces" class has been removed from N-Caltech101 to avoid confusion, leaving 100 object classes plus a background class. The N-Caltech101 dataset was captured by mounting the ATIS sensor on a motorized pan-tilt unit and having the sensor move while it views Caltech101 examples on an LCD monitor as shown in the video below. A full description of the dataset and how it was created can be found in the paper below. Please cite this paper if you make use of the dataset.

Comprises 11 hand gesture categories from 29 subjects under 3 illumination conditions. Source: A Low Power, Fully Event-Based Gesture Recognition System

A large real-world event-based dataset for object classification. Source: HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification

The SUN Attribute dataset consists of 14,340 images from 717 scene categories, and each category is annotated with a taxonomy of 102 discriminate attributes. The dataset can be used for high-level scene understanding and fine-grained scene recognition. Source: Zero-Shot Learning with Multi-Battery Factor Analysis Image Source: https://cs.brown.edu/~gmpatter/sunattributes.html

The SHREC dataset contains 14 dynamic gestures performed by 28 participants (all participants are right handed) and captured by the Intel RealSense short range depth camera. Each gesture is performed between 1 and 10 times by each participant in two way: using one finger and the whole hand. Therefore, the dataset is composed by 2800 sequences captured. The depth image, with a resolution of 640x480, and the coordinates of 22 joints (both in the 2D depth image space and in the 3D world space) are saved for each frame of each sequence in the dataset. Source: Exploiting Recurrent Neural Networks and Leap Motion Controller for Sign Language and Semaphoric Gesture Recognition Image Source: http://tosca.cs.technion.ac.il/book/shrec.html

Developing robot perception systems for handling objects in the real-world requires computer vision algorithms to be carefully scrutinized with respect to the expected operating domain. This demands large quantities of ground truth data to rigorously evaluate the performance of algorithms. The Object Cluttered Indoor Dataset is an RGBD-dataset containing point-wise labeled point-clouds for each object. The data was captured using two ASUS-PRO Xtion cameras that are positioned at different heights. It captures diverse settings of objects, background, context, sensor to scene distance, viewpoint angle and lighting conditions. The main purpose of OCID is to allow systematic comparison of existing object segmentation methods in scenes with increasing amount of clutter. In addition OCID does also provide ground-truth data for other vision tasks like object-classification and recognition. Source: OCID

CIFAR10-DVS is an event-stream dataset for object classification. 10,000 frame-based images that come from CIFAR-10 dataset are converted into 10,000 event streams with an event-based sensor, whose resolution is 128×128 pixels. The dataset has an intermediate difficulty with 10 different classes. The repeated closed-loop smooth (RCLS) movement of frame-based images is adopted to implement the conversion. Due to the transformation, they produce rich local intensity changes in continuous time which are quantized by each pixel of the event-based camera. Source: Structure-Aware Network for Lane Marker Extraction with Dynamic Vision Sensor Image Source: https://www.frontiersin.org/articles/10.3389/fnins.2017.00309/full

Includes accurate pixel-wise motion masks, egomotion and ground truth depth. Source: EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras

The MECCANO dataset is the first dataset of egocentric videos to study human-object interactions in industrial-like settings. The MECCANO dataset has been acquired in an industrial-like scenario in which subjects built a toy model of a motorbike. We considered 20 object classes which include the 16 classes categorizing the 49 components, the two tools (screwdriver and wrench), the instructions booklet and a partial_model class. Additional details related to the MECCANO: 20 different subjects in 2 countries (IT, U.K.) Video Acquisition: 1920x1080 at 12.00 fps 11 training videos and 9 validation/test videos 8857 video segments temporally annotated indicating the verbs which describe the actions performed 64349 active objects annotated with bounding boxes 12 verb classes, 20 objects classes and 61 action classes

The N-ImageNet dataset is an event-camera counterpart for the ImageNet dataset. The dataset is obtained by moving an event camera around a monitor displaying images from ImageNet. N-ImageNet contains approximately 1,300k training samples and 50k validation samples. In addition, the dataset also contains variants of the validation dataset recorded under a wide range of lighting or camera trajectories. Additional details about the dataset are explained in the paper available through this link. Please cite this paper if you make use of the dataset.

Washington RGB-D is a widely used testbed in the robotic community, consisting of 41,877 RGB-D images organized into 300 instances divided in 51 classes of common indoor objects (e.g. scissors, cereal box, keyboard etc). Each object instance was positioned on a turntable and captured from three different viewpoints while rotating. Source: Learning Deep Visual Object Models From Noisy Web Data: How to Make it Work Image Source: https://rgbd-dataset.cs.washington.edu/

DeepScores contains high quality images of musical scores, partitioned into 300,000 sheets of written music that contain symbols of different shapes and sizes. For advancing the state-of-the-art in small objects recognition, and by placing the question of object recognition in the context of scene understanding. Source: DeepScores -- A Dataset for Segmentation, Detection and Classification of Tiny Objects Image Source: https://tuggeluk.github.io/deepscores/

TUM-GAID (TUM Gait from Audio, Image and Depth) collects 305 subjects performing two walking trajectories in an indoor environment. The first trajectory is traversed from left to right and the second one from right to left. Two recording sessions were performed, one in January, where subjects wore heavy jackets and mostly winter boots, and another one in April, where subjects wore lighter clothes. The action is captured by a Microsoft Kinect sensor which provides a video stream with a resolution of 640×480 pixels and a frame rate around 30 FPS. Source: Energy-based Tuning of Convolutional Neural Networks on Multi-GPUs Image Source: https://www.ei.tum.de/mmk/verschiedenes/tum-gaid-database/

APRICOT is a collection of over 1,000 annotated photographs of printed adversarial patches in public locations. The patches target several object categories for three COCO-trained detection models, and the photos represent natural variation in position, distance, lighting conditions, and viewing angle. Source: APRICOT: A Dataset of Physical Adversarial Attacks on Object Detection Image Source: https://apricot.mitre.org/

The RIT-18 dataset was built for the semantic segmentation of remote sensing imagery. It was collected with the Tetracam Micro-MCA6 multispectral imaging sensor flown on-board a DJI-1000 octocopter. The features this dataset include 1) very-high resolution multispectral imagery from a drone, 2) six-spectral VNIR bands, and 3) 18 object classes (plus background) with a severely unbalanced class distribution. Source: Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning Image Source: https://github.com/rmkemker/RIT-18

Freiburg Groceries is a groceries classification dataset consisting of 5000 images of size 256x256, divided into 25 categories. It has imbalanced class sizes ranging from 97 to 370 images per class. Images were taken in various aspect ratios and padded to squares. Source: XNAS: Neural Architecture Search with Expert Advice Image Source: http://aisdatasets.informatik.uni-freiburg.de/freiburg_groceries_dataset/

iCubWorld datasets are collections of images recording the visual experience of iCub while observing objects in its typical environment, a laboratory or an office. The acquisition setting is devised to allow a natural human-robot interaction, where a teacher verbally provides the label of the object of interest and shows it to the robot, by holding it in the hand; the iCub can either track the object while the teacher moves it, or take it in its hand.

A large-scale video dataset for MOR in aerial videos. Source: MOR-UAV: A Benchmark Dataset and Baselines for Moving Object Recognition in UAV Videos

ARID is a large-scale, multi-view object dataset collected with an RGB-D camera mounted on a mobile robot. Source: ARID

Cube++ is a novel dataset for the color constancy problem that continues on the Cube+ dataset. It includes 4890 images of different scenes under various conditions. For calculating the ground truth illumination, a calibration object with known surface colors was placed in every scene. Source: https://github.com/Visillect/CubePlusPlus Image Source: https://github.com/Visillect/CubePlusPlus

CURE-OR is a large-scale, controlled, and multi-platform object recognition dataset denoted as Challenging Unreal and Real Environments for Object Recognition. In this dataset, there are 1,000,000 images of 100 objects with varying size, color, and texture that are positioned in five different orientations and captured using five devices including a webcam, a DSLR, and three smartphone cameras in real-world (real) and studio (unreal) environments. The controlled challenging conditions include underexposure, overexposure, blur, contrast, dirty lens, image noise, resizing, and loss of color information.

The NYU Symmetry database contains 176 single-symmetry and 63 multiple-symmetry images (.png files) with accompanying ground-truth annotations (.mat files). Also included are a .m file to visualize the annotations on top of the images, and a .txt file with instructions on how to interpret the .mat annotations.

The Few-Shot Object Learning (FewSOL) dataset can be used for object recognition with a few images per object. It contains 336 real-world objects with 9 RGB-D images per object from different views. Object segmentation masks, object poses and object attributes are provided. In addition, synthetic images generated using 330 3D object models are used to augment the dataset. FewSOL dataset can be used to study a set of few-shot object recognition problems such as classification, detection and segmentation, shape reconstruction, pose estimation, keypoint correspondences and attribute recognition. Motivation: If robots can recognize objects from a few exemplar images, it is possible to scale up the number of objects a robot can recognize because collecting a few images per object is a much easier process compared to building a 3D model of an object. In addition, models trained in the meta-learning setting can generalize to new objects without re-training.

DOORS is a dataset designed for boulders recognition, centroid regression, segmentation, and navigation applications. The dataset is divided into two sets: - Regression: Contains images, masks, and labels for 4 splits of single boulders positioned on the surface of a spherical mesh. It can be used to perform navigation, boulder recognition, segmentation, and centroid regression. - Segmentation: Contain images, masks, and labels of 2 datasets: DS1 and DS2. DS1 is made of the same images of the Regression dataset but is specifically designed for segmentation. DS2 is made of images with multiple instances of boulders appearing on the surface of the Didymos asteroid model Source: DOORS: Dataset fOr bOuldeRs Segmentation. Statistical properties and Blender setup Image Source: https://arxiv.org/pdf/2210.16253v1.pdf

HASY is a dataset of single symbols similar to MNIST. It contains 168,233 instances of 369 classes. HASY contains two challenges: A classification challenge with 10 pre-defined folds for 10-fold cross-validation and a verification challenge. Source: The HASYv2 dataset

A new large-scale retail product dataset for fine-grained image classification. Unlike previous datasets focusing on relatively few products, more than 500,000 images of retail products on shelves were collected, belonging to 2000 different products. The dataset aims to advance the research in retail object recognition, which has massive applications such as automatic shelf auditing and image-based product information retrieval. Source: RP2K: A Large-Scale Retail Product Dataset for Fine-Grained Image Classification

Egoshots is a 2-month Ego-vision Dataset with Autographer Wearable Camera annotated "for free" with transfer learning. Three state of the art pre-trained image captioning models are used. The dataset represents the life of 2 interns while working at Philips Research (Netherlands) (May-July 2015) generously donating their data. Source: https://github.com/NataliaDiaz/Egoshots

UW Indoor Scenes (UW-IS) Occluded dataset is curated using commodity hardware (Intel RealSense D435) to reflect real world robotics scenarios. It consists of two completely different indoor environments. The first environment is a lounge where the objects are placed on a tabletop. The second environment is a mock warehouse setup where the objects are placed on a shelf. For each of these environments, we have RGB-D images from 36 videos comprising five to seven objects each, taken from distances up to approximately 2m. The videos cover two different lighting conditions, three different levels of object separation for three different object categories (i.e., kitchen objects, food items, and tools/miscellaneous). The first level of object separation is such that there is no object occlusion. The second level of object separation is such that some occlusion occurs, while the third level is where the objects are placed extremely close together. Overall, the dataset considers 20 object classes and consists of 8,456 images, which have a total of 42,902 object instances. We also provide instance segmentation masks and 6D pose annotations for all the images generated using LabelFusion (Marion et al., 2018)

UW-IS (UW Indoor Scenes) is a dataset for object recognition in indoor environments comprising scene images from two different environments, namely, a living room and a mock warehouse.

The ARC-100 dataset was collected as part of a prototype retail checkout system titled ARC (Automatic Retail Checkout). It consists of 31,000 $640\times480$ RGB images of 100 commonly found retail items in Lahore, Pakistan. Each retail item has 310 images captured at various logical orientations (on a black, matte finish conveyor belt) by a Logitech C310 webcam, under a wooden hood frame illuminated by LED strips (luminance set to approximately $70lx$). In the proposed setup, images were pre-processed and standardized before feeding into a Convolutional Neural Network for identification. Links: - ARC paper - ARC-100 dataset

Multimodal object recognition is still an emerging field. Thus, publicly available datasets are still rare and of small size. This dataset was developed to help fill this void and presents multimodal data for 63 objects with some visual and haptic ambiguity. The dataset contains visual, kinesthetic and tactile (audio/vibrations) data. To completely solve sensory ambiguity, sensory integration/fusion would be required. This report describes the creation and structure of the dataset. The first section explains the underlying approach used to capture the visual and haptic properties of the objects. The second section describes the technical aspects (experimental setup) needed for the collection of the data. The third section introduces the objects, while the final section describes the structure and content of the dataset.