2D Semantic Segmentation

The Stanford Background dataset contains 715 RGB images and the corresponding label images. Images are approximately 240×320 pixels in size and pixels are classified into eight different categories Source: Unsupervised Total Variation Loss for Semi-supervised Deep Learning of Semantic Segmentation Image Source: http://dags.stanford.edu/projects/scenedataset.html

A novel dataset and benchmark, which features 1482 RGB-D scans of 478 environments across multiple time steps. Each scene includes several objects whose positions change over time, together with ground truth annotations of object instances and their respective 6DoF mappings among re-scans. Source: RIO: 3D Object Instance Re-Localization in Changing Indoor Environments

The PanoContext dataset contains 500 annotated cuboid layouts of indoor environments such as bedrooms and living rooms. Source: LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image Image Source: https://panocontext.cs.princeton.edu/paper.pdf

KITTI Road is road and lane estimation benchmark that consists of 289 training and 290 test images. It contains three different categories of road scenes: * uu - urban unmarked (98/100) * um - urban marked (95/96) * umm - urban multiple marked lanes (96/94) * urban - combination of the three above Ground truth has been generated by manual annotation of the images and is available for two different road terrain types: road - the road area, i.e, the composition of all lanes, and lane - the ego-lane, i.e., the lane the vehicle is currently driving on (only available for category "um"). Ground truth is provided for training images only. Source: http://www.cvlibs.net/datasets/kitti/eval_road.php Image Source: http://www.cvlibs.net/datasets/kitti/eval_road.php

AID is a new large-scale aerial image dataset, by collecting sample images from Google Earth imagery. Note that although the Google Earth images are post-processed using RGB renderings from the original optical aerial images, it has proven that there is no significant difference between the Google Earth images with the real optical aerial images even in the pixel-level land use/cover mapping. Thus, the Google Earth images can also be used as aerial images for evaluating scene classification algorithms. The new dataset is made up of the following 30 aerial scene types: airport, bare land, baseball field, beach, bridge, center, church, commercial, dense residential, desert, farmland, forest, industrial, meadow, medium residential, mountain, park, parking, playground, pond, port, railway station, resort, river, school, sparse residential, square, stadium, storage tanks and viaduct. All the images are labelled by the specialists in the field of remote sensing image interpretation, and some samples of each class are shown in Fig.1. In all, the AID dataset has a number of 10000 images within 30 classes. The images in AID are actually multi-source, as Google Earth images are from different remote imaging sensors. This brings more challenges for scene classification than the single source images like UC-Merced dataset. Moreover, all the sample images per each class in AID are carefully chosen from different countries and regions around the world, mainly in China, the United States, England, France, Italy, Japan, Germany, etc., and they are extracted at different time and seasons under different imaging conditions, which increases the intra-class diversities of the data.

TextOCR is a dataset to benchmark text recognition on arbitrary shaped scene-text. TextOCR requires models to perform text-recognition on arbitrary shaped scene-text present on natural images. TextOCR provides ~1M high quality word annotations on TextVQA images allowing application of end-to-end reasoning on downstream tasks such as visual question answering or image captioning. Dataset statistics: - 28,134 natural images from TextVQA - 903,069 annotated scene-text words - 32 words per image on average

3DSSG provides 3D semantic scene graphs for 3RScan. A semantic scene graph is defined by a set of tuples between nodes and edges where nodes represent specific 3D object instances in a 3D scan. Nodes are defined by its semantics, a hierarchy of classes as well as a set of attributes that describe the visual and physical appearance of the object instance and their affordances. The edges in our graphs are the semantic relationships (predicates) between the nodes such as standing on, hanging on, more comfortable than or same material.

The Street View Text (SVT) dataset was harvested from Google Street View. Image text in this data exhibits high variability and often has low resolution. In dealing with outdoor street level imagery, we note two characteristics. (1) Image text often comes from business signage and (2) business names are easily available through geographic business searches. These factors make the SVT set uniquely suited for word spotting in the wild: given a street view image, the goal is to identify words from nearby businesses. Note: the dataset has undergone revision since the time it was evaluated in this publication. Please consult the ICCV2011 paper for most up-to-date results. Source: Street View Text Dataset Image source: Street View Text Dataset

A Large-scale Dataset for Video Scene Parsing in the Wild

The Segmentation of Underwater IMagery (SUIM) dataset contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborative experiments, and annotated by human participants. Source: Semantic Segmentation of Underwater Imagery: Dataset and Benchmark Image Source: http://irvlab.cs.umn.edu/resources/suim-dataset

The Crowd Instance-level Human Parsing (CIHP) dataset has 38,280 diverse human images. Each image in CIHP is labeled with pixel-wise annotations on 20 categories and instance-level identification. The dataset can be used for the human part segmentation task. Source: Parsing R-CNN for Instance-Level Human Analysis Image Source: https://arxiv.org/abs/1808.00157

The Multi-Object and Segmentation (MOTS) benchmark [2] consists of 21 training sequences and 29 test sequences. It is based on the KITTI Tracking Evaluation 2012 and extends the annotations to the Multi-Object and Segmentation (MOTS) task. To this end, we added dense pixel-wise segmentation labels for every object. We evaluate submitted results using the metrics HOTA, CLEAR MOT, and MT/PT/ML. We rank methods by HOTA [1]. Our development kit and GitHub evaluation code provide details about the data format as well as utility functions for reading and writing the label files. (adapted for the segmentation case). Evaluation is performed using the code from the TrackEval repository. [1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020. [2] P. Voigtlaender, M. Krause, A. Os̆ep, J. Luiten, B. Sekar, A. Geiger, B. Leibe: MOTS: Multi-Object Tracking and Segmentation. CVPR 2019.

PSG dataset has 48749 images with 133 object classes (80 objects and 53 stuff) and 56 predicate classes. It annotates inter-segment relations based on COCO panoptic segmentation.

The SensatUrbat dataset is an urban-scale photogrammetric point cloud dataset with nearly three billion richly annotated points, which is five times the number of labeled points than the existing largest point cloud dataset. The dataset consists of large areas from two UK cities, covering about 6 km^2 of the city landscape. In the dataset, each 3D point is labeled as one of 13 semantic classes, such as ground, vegetation, car, etc.. Source: https://github.com/QingyongHu/SensatUrban Image Source: https://github.com/QingyongHu/SensatUrban

RADIATE (RAdar Dataset In Adverse weaThEr) is new automotive dataset created by Heriot-Watt University which includes Radar, Lidar, Stereo Camera and GPS/IMU. The data is collected in different weather scenarios (sunny, overcast, night, fog, rain and snow) to help the research community to develop new methods of vehicle perception. The radar images are annotated in 7 different scenarios: Sunny (Parked), Sunny/Overcast (Urban), Overcast (Motorway), Night (Motorway), Rain (Suburban), Fog (Suburban) and Snow (Suburban). The dataset contains 8 different types of objects (car, van, truck, bus, motorbike, bicycle, pedestrian and group of pedestrians). Source: https://github.com/marcelsheeny/radiate_sdk Image Source: https://github.com/marcelsheeny/radiate_sdk

Toronto-3D is a large-scale urban outdoor point cloud dataset acquired by an MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of road and consists of about 78.3 million points. Point clouds has 10 attributes and classified in 8 labelled object classes. Source: https://github.com/WeikaiTan/Toronto-3D Image Source: https://github.com/WeikaiTan/Toronto-3D

A simulation-based dataset featuring 20,000 stack configurations composed of a variety of elementary geometric primitives richly annotated regarding semantics and structural stability. Source: ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking

Features a large-scale dataset with 12,263 annotated images. Two tasks, namely text localization and end-to-end recognition, are set up. The competition took place from January 20 to May 31, 2017. 23 valid submissions were received from 19 teams. Source: ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)

Multimodal material segmentation (MCubeS) dataset contains 500 sets of images from 42 street scenes. Each scene has images for four modalities: RGB, angle of linear polarization (AoLP), degree of linear polarization (DoLP), and near-infrared (NIR). The dataset provides annotated ground truth labels for both material and semantic segmentation for every pixel. The dataset is divided training set with 302 image sets, validation set with 96 image sets, and test set with 102 image sets. Each image has 1224 x 1024 pixels and a total of 20 class labels per pixel.

Touchdown is a corpus for executing navigation instructions and resolving spatial descriptions in visual real-world environments. The task is to follow instruction to a goal position and there find a hidden object, Touchdown the bear. Source: https://github.com/lil-lab/touchdown Image Source: https://github.com/lil-lab/touchdown

The heavily occluded scene text (HOST) dataset is a dataset that contains images of text with occlusions. It is used to improve the recognition performance of occluded text in machine vision applications 1. The dataset is composed of 4832 images that are manually occluded in weak or heavy degrees.

We propose a large-scale benchmark here, which contains a total of 6,461 mirror images with ground truth annotations.

The CUTE80 dataset is a lightweight collection of images specifically designed for text detection in natural scene images. It contains a total of 13,000 annotated page images across five different popular categories: 1) Table 2) Figure 3) Natural image 4) Logo 5) ignature

MLRSNet is a a multi-label high spatial resolution remote sensing dataset for semantic scene understanding. It provides different perspectives of the world captured from satellites. That is, it is composed of high spatial resolution optical satellite images. MLRSNet contains 109,161 remote sensing images that are annotated into 46 categories, and the number of sample images in a category varies from 1,500 to 3,000. The images have a fixed size of 256×256 pixels with various pixel resolutions (~10m to 0.1m). Moreover, each image in the dataset is tagged with several of 60 predefined class labels, and the number of labels associated with each image varies from 1 to 13. The dataset can be used for multi-label based image classification, multi-label based image retrieval, and image segmentation. Source: https://github.com/cugbrs/MLRSNet Image Source: Qi et al

SceneNet is a dataset of labelled synthetic indoor scenes. There are several labeled indoor scenes, including: - 11 Bedroom scenes with 428 objects - 15 Office scenes with 1,203 objects - 11 Kitchen scenes with 797 objects - 10 Living Room scenes with 715 objects - 10 Bathrooms with 556 objects Source: https://robotvault.bitbucket.io/ Image Source: https://robotvault.bitbucket.io/big_scene.html

The Weakly Occluded Scene Text (WOST) dataset is a public dataset for scene text segmentation. It is used to generate pixel-level annotations in scene text images 1. The dataset is designed to contain weakly annotated images, which means that the images are not fully annotated with pixel-level labels.

DADA-2000 is a large-scale benchmark with 2000 video sequences (named as DADA-2000) is contributed with laborious annotation for driver attention (fixation, saccade, focusing time), accident objects/intervals, as well as the accident categories, and superior performance to state-of-the-arts are provided by thorough evaluations. Source: DADA: A Large-scale Benchmark and Model for Driver Attention Prediction in Accidental Scenarios Image Source: Fang et al

TextSeg is a large-scale fine-annotated and multi-purpose text detection and segmentation dataset, collecting scene and design text with six types of annotations: word- and character-wise bounding polygons, masks and transcriptions. Source: https://github.com/SHI-Labs/Rethinking-Text-Segmentation Image Source: https://github.com/SHI-Labs/Rethinking-Text-Segmentation

A novel benchmark dataset that includes a manually annotated point cloud for over 260 million laser scanning points into 100'000 (approx.) assets from Dublin LiDAR point cloud [12] in 2015. Objects are labelled into 13 classes using hierarchical levels of detail from large (i.e., building, vegetation and ground) to refined (i.e., window, door and tree) elements. Source: DublinCity: Annotated LiDAR Point Cloud and its Applications

ISTD+ consists of shadow images, shadow-free images, and shadow masks, with 1,330 training images and 540 testing images from 135 unique background scenes. ISTD suffers from color and luminosity inconsistencies between shadow and shadow-free images, which ISTD+ corrects with a color compensation mechanism to ensure uniform pixel colors across the ground-truth images.

This dataset includes 4,500 fully annotated images (over 30,000 license plate characters) from 150 vehicles in real-world scenarios where both the vehicle and the camera (inside another vehicle) are moving. The images were acquired with three different cameras and are available in the Portable Network Graphics (PNG) format with a size of 1,920 × 1,080 pixels. The cameras used were: GoPro Hero4 Silver, Huawei P9 Lite, and iPhone 7 Plus. We collected 1,500 images with each camera, divided as follows: - 900 of cars with gray license plates; - 300 of cars with red license plates; - 300 of motorcycles with gray license plates. The dataset is split as follows: 40% for training, 40% for testing and 20% for validation. Every image has the following annotations available in a text file: the camera in which the image was taken, the vehicle’s position and information such as type (car or motorcycle), manufacturer, model and year; the identification and position of the license plate, as well as the position of its characters. Source: A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector

The IIIT5K dataset contains 5,000 text instance images: 2,000 for training and 3,000 for testing. It contains words from street scenes and from originally-digital images. Every image is associated with a 50 -word lexicon and a 1,000 -word lexicon.

The MHP dataset contains multiple persons captured in real-world scenes with pixel-level fine-grained semantic annotations in an instance-aware setting. Source: Multiple-Human Parsing in the Wild Image Source: Li et al

SVTP dataset stands for Scene Text Recognition Datasets. It is a collection of 4 popular Latin/English scene text recognition datasets, namely IIIT5K, SVT, SVTP, and CUTE-80. These datasets only provide case-insensitive annotations and no punctuation marks.

A Multi-Task 4D Radar-Camera Fusion Dataset for Autonomous Driving on Water Surfaces description of the dataset * WaterScenes, the first multi-task 4D radar-camera fusion dataset on water surfaces, which offers data from multiple sensors, including a 4D radar, monocular camera, GPS, and IMU. It can be applied in multiple tasks, such as object detection, instance segmentation, semantic segmentation, free-space segmentation, and waterline segmentation. * Our dataset covers diverse time conditions (daytime, nightfall, night), lighting conditions (normal, dim, strong), weather conditions (sunny, overcast, rainy, snowy) and waterway conditions (river, lake, canal, moat). An information list is also offered for retrieving specific data for experiments under different conditions. * We provide 2D box-level and pixel-level annotations for camera images, and 3D point-level annotations for radar point clouds. We also offer precise timestamps for the synchronization of different sensors, as well as intrinsic and extrinsic parameters. * We provide a toolkit for radar point clouds that includes: pre-processing, labeling, projection and visualization, assisting researchers in processing and analyzing our dataset.

A new and challenging video database of dynamic scenes that more than doubles the size of those previously available. This dataset is explicitly split into two subsets of equal size that contain videos with and without camera motion to allow for systematic study of how this variable interacts with the defining dynamics of the scene per se. Source: Temporal Residual Networks for Dynamic Scene Recognition