3D Instance Segmentation

ScanNet is an instance-level indoor RGB-D dataset that includes both 2D and 3D data. It is a collection of labeled voxels rather than points or objects. Up to now, ScanNet v2, the newest version of ScanNet, has collected 1513 annotated scans with an approximate 90% surface coverage. In the semantic segmentation task, this dataset is marked in 20 classes of annotated 3D voxelized objects. Source: A Review of Point Cloud Semantic Segmentation Image Source: http://www.scan-net.org/

The Stanford 3D Indoor Scene Dataset (S3DIS) dataset contains 6 large-scale indoor areas with 271 rooms. Each point in the scene point cloud is annotated with one of the 13 semantic categories. Source: Grid-GCN for Fast and Scalable Point Cloud Learning Image Source: https://www.researchgate.net/figure/Examples-of-classified-scenes-in-S3DIS-dataset-left-with-groundtruth-right_fig2_328307943

KITTI-360 is a large-scale dataset that contains rich sensory information and full annotations. It is the successor of the popular KITTI dataset, providing more comprehensive semantic/instance labels in 2D and 3D, richer 360 degree sensory information (fisheye images and pushbroom laser scans), very accurate and geo-localized vehicle and camera poses, and a series of new challenging benchmarks.

PartNet is a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information. The dataset consists of 573,585 part instances over 26,671 3D models covering 24 object categories. This dataset enables and serves as a catalyst for many tasks such as shape analysis, dynamic 3D scene modeling and simulation, affordance analysis, and others. Source: PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding Image Source: https://cs.stanford.edu/~kaichun/partnet/

SceneNN is an RGB-D scene dataset consisting of more than 100 indoor scenes. The scenes are captured at various places, e.g., offices, dormitory, classrooms, pantry, etc., from University of Massachusetts Boston and Singapore University of Technology and Design. All scenes are reconstructed into triangle meshes and have per-vertex and per-pixel annotation. The dataset is additionally enriched with fine-grained information such as axis-aligned bounding boxes, oriented bounding boxes, and object poses. Source: SceneNN: A Scene Meshes Dataset with aNNotations Image Source: http://103.24.77.34/scenenn/home/

The ScanNet200 benchmark studies 200-class 3D semantic segmentation - an order of magnitude more class categories than previous 3D scene understanding benchmarks. The source of scene data is identical to ScanNet, but parses a larger vocabulary for semantic and instance segmentation

Our project (STPLS3D) aims to provide a large-scale aerial photogrammetry dataset with synthetic and real annotated 3D point clouds for semantic and instance segmentation tasks. Although various 3D datasets with different functions and scales have been proposed recently, it remains challenging for individuals to complete the whole pipeline of large-scale data collection, sanitization, and annotation (e.g., semantic and instance labels). Moreover, the created datasets usually suffer from extremely imbalanced class distribution or partial low-quality data samples. Motivated by this, we explore the procedurally synthetic 3D data generation paradigm to equip individuals with the full capability of creating large-scale annotated photogrammetry point clouds. Specifically, we introduce a synthetic aerial photogrammetry point clouds generation pipeline that takes full advantage of open geospatial data sources and off-the-shelf commercial packages. Unlike generating synthetic data in virtual games, where the simulated data usually have limited gaming environments created by artists, the proposed pipeline simulates the reconstruction process of the real environment by following the same UAV flight pattern on a wide variety of synthetic terrain shapes and building densities, which ensure similar quality, noise pattern, and diversity with real data. In addition, the precise semantic and instance annotations can be generated fully automatically, avoiding the expensive and time-consuming manual annotation process. Based on the proposed pipeline, we present a richly-annotated synthetic 3D aerial photogrammetry point cloud dataset, termed STPLS3D, with more than 16 km^2 of landscapes and up to 18 fine-grained semantic categories. For verification purposes, we also provide a parallel dataset collected from four areas in the real environment. Source: https://github.com/meidachen/STPLS3D Image source: https://github.com/meidachen/STPLS3D/blob/main/imgs/STPLS3D.png

ScanNet++ is a large scale dataset with 450+ 3D indoor scenes containing sub-millimeter resolution laser scans, registered 33-megapixel DSLR images, and commodity RGB-D streams from iPhone. The 3D reconstructions are annotated with long-tail and label-ambiguous semantics to benchmark semantic understanding methods, while the coupled DSLR and iPhone captures enable benchmarking of novel view synthesis methods in high-quality and commodity settings.

The challenge of accurately segmenting individual trees from laser scanning data hinders the assessment of crucial tree parameters necessary for effective forest management, impacting many downstream applications. While dense laser scanning offers detailed 3D representations, automating the segmentation of trees and their structures from point clouds remains difficult. The lack of suitable benchmark datasets and reliance on small datasets have limited method development. The emergence of deep learning models exacerbates the need for standardized benchmarks. Addressing these gaps, the FOR-instance data represent a novel benchmarking dataset to enhance forest measurement using dense airborne laser scanning data, aiding researchers in advancing segmentation methods for forested 3D scenes. In this repository, users will find forest laser scanning point clouds from unamnned aerial vehicle (using Riegl sensors) that are manually segmented according to the individual trees (1130 trees) and semantic classes. The point clouds are subdivided into five data collections representing different forests in Norway, the Czech Republic, Austria, New Zealand, and Australia. These data are meant to be used either for developement of new methods (using the dev data) or for testing of exisitng methods (test data). The data splits are provided in the data_split_metadata.csv file. A full description of the FOR-instance data can be found at http://arxiv.org/abs/2309.01279

MICCAI Challenge on Circuit Reconstruction from Electron Microscopy Images. # About The goal of this challenge is to evaluate algorithms for automatic reconstruction of neurons and neuronal connectivity from serial section electron microscopy data. The comparison is performed not only by evaluating the quality of neuron segmentations, but also by assessing the accuracy of detecting synapses and identifying synaptic partners. The challenge is carried out on three large and diverse datasets from adult Drosophila melanogaster brain tissue, comprising neuron segmentation ground truth and annotations for synaptic connections. A successful solution would demonstrate its efficiency and generalizability, and carry great potential to reduce the time spent on manual reconstruction of neural circuits in electron microscopy volumes. # Description We provide three datasets, each consisting of two (5 μm)3 volumes (training and testing, each 1250 px × 1250 px × 125 px) of serial section EM of the adult fly brain. Each volume has neuron and synapse labelings and annotations for pre- and post-synaptic partners.

We introduce MultiScan, a scalable RGBD dataset construction pipeline leveraging commodity mobile devices to scan indoor scenes with articulated objects and web-based semantic annotation interfaces to efficiently annotate object and part semantics and part mobility parameters. We use this pipeline to collect 273 scans of 117 indoor scenes containing 10957 objects and 5129 parts. The resulting MultiScan dataset provides RGBD streams with per-frame camera poses, textured 3D surface meshes, richly annotated part-level and object-level semantic labels, and part mobility parameters. We validate our dataset on instance segmentation and part mobility estimation tasks and benchmark methods for these tasks from prior work. Our experiments show that part segmentation and mobility estimation in real 3D scenes remain challenging despite recent progress in 3D object segmentation.

Contains mitochondria instances. Source: MitoEM

Understanding and analyzing animal behavior is increasingly essential to protect endangered animal species. However, the application of advanced computer vision techniques in this regard is minimal, which boils down to lacking large and diverse datasets for training deep models. To break the deadlock, we present LoTE-Animal, a large-scale endangered animal dataset collected over 12 years, to foster the application of deep learning in rare species conservation. The collected data contains vast variations such as ecological seasons, weather conditions, periods, viewpoints, and habitat scenes. So far, we retrieved at least 500K videos and 1.2 million images. Specifically, we selected and annotated 11 endangered animals for behavior understanding, including 10K video sequences for the action recognition task, 28K images for object detection, instance segmentation, and pose estimation tasks. In addition, we gathered 7K web images of the same species as source domain data for the domain adaptation task. We provide evaluation results of representative vision understanding approaches and cross-domain experiments. LoTE-Animal dataset would facilitate the community to research more advanced machine learning models and explore new tasks to aid endangered animal conservation. Our dataset will be released with the paper.

The 'Me 163' was a Second World War fighter airplane and a result of the German air force secret developments. One of these airplanes is currently owned and displayed in the historic aircraft exhibition of the 'Deutsches Museum' in Munich, Germany. To gain insights with respect to its history, design and state of preservation, a complete CT scan was obtained using an industrial XXL-computer tomography scanner at Fraunhofer EZRT . Using the CT data from the Me 163, all its details can visually be examined at various levels, ranging from the complete hull down to single sprockets and rivets. However, while a trained human observer can identify and interpret the volumetric data with all its parts and connections, a virtual dissection of the airplane and all its different parts would be quite desirable. Nevertheless, this means, that an instance segmentation of all components and objects of interest into disjoint entities from the CT data is necessary. As of currently, no adequate computer-assisted tools for automated or semi-automated segmentation of such XXL-airplane data are available, in a first step, an interactive data annotation and object labelling process has been established. So far, seven $512 \times 512 \times 512$ voxel sub-volumes from the Me 163 airplane have been annotated and labelled, whose results can potentially be used for various new applications in the field of digital heritage, non-destructive testing, or machine-learning.

Heritage Pointcloud Instance Collection dataset, acquired from two large buildings and annotated at a point-wise semantic level based on existent BIM models. Devid Campagnolo, Elena Camuffo, Umberto Michieli, Paolo Borin, Simone Milani and Andrea Giordano, "Fully Automated Scan-to-BIM via Point Cloud Instance Segmentation", In Proceedings of the International Conference on Image Processing (ICIP) 2023.

XA Bin-Picking is a point-cloud dataset comprising both simulated and real-world scenes with three industrial parts. Synthesized scenes consists of 1000 training samples. The test samples are real scenes and the ground truth instance labels are made manually. There are 20 to 30 identical types of parts randomly piled up in a scene. Each scene contains about 60,000 boundary points. Each point in the scene has instance annotations. The parts are texture-less and have no discernible color. Both of training samples and test sam- ples only contain the boundary points of parts. Source: A Convolutional Neural Network for Point Cloud Instance Segmentation in Cluttered Scene Trained by Synthetic Data Without Color Image Source: A Convolutional Neural Network for Point Cloud Instance Segmentation in Cluttered Scene Trained by Synthetic Data Without Color

InfiniteRep is a synthetic, open-source dataset for fitness and physical therapy (PT) applications. It includes 1k videos of diverse avatars performing multiple repetitions of common exercises. It includes significant variation in the environment, lighting conditions, avatar demographics, and movement trajectories. From cadence to kinematic trajectory, each rep is done slightly differently -- just like real humans. InfiniteRep videos are accompanied by a rich set of pixel-perfect labels and annotations, including frame-specific repetition counts. The dataset features: + 100 videos per exercise, spanning 5 to 10 repetitions each (1,000 videos total) + 7 unique indoor scenes + Realistic environmental occlusion (+ corresponding labels) + Diverse lighting conditions + Varied body shape, skin tones, and clothing + Rich annotations for 2D and 3D supervision ## Exercises The dataset currently includes the following exercises: + Pushups + Alternating Bicep Curls (with dumbbells) + Delt Flys (with dumbbells) + Squats + Bird Dogs + Supermans + Bicycle Crunches + Leg Raises + Front Raises (with dumbbells) + Overhead Press (with dumbbells) ## Annotations The dataset includes the following annotations: + Bounding boxes + Segmentation masks + Keypoints + Joint angles (quaternions) + Percent occlusion + Avatar characteristics + Camera position + and more Want depth labels? They are not included in the dataset but we can send them to you. Email us at info@toinfinity.ai. ## Download Download the dataset: toinfinity.ai/infiniterep Github repo with additional documentation: https://github.com/toinfinityai/InfiniteRep ## Need more data? Infinity AI specializes in generating custom synthetic data. If you need more (or different data), drop us a line at info@toinfinity.ai (we read every email).

This dataset called Indoor Lodz University of Technology Point Cloud Dataset (InLUT3D) is a point cloud set tailored for real object classification and both semantic and instance segmentation tasks. Comprising of 321 scans, some areas in the dataset are covered by multiple scans. All of them are captured using the Leica BLK360 scanner. The points are divided into 18 distinct categories outlined in the label.yaml file along with their respective codes and colors. Among categories you will find: - ceiling, - floor, - wall, - stairs, - column, - chair, - sofa, - table, - storage, - door, - window, - plant, - dish, - wallmounted, - device, - radiator, - lighting, - other. Several challenges are intrinsic to the presented dataset: 1. Extremely non-uniform categories distribution across the dataset. 1. Presence of virtual images, particularly in reflective surfaces, and data exterior to windows and doors. 1. Occurrence of missing data due to scanning shadows (certain areas were inaccessible to the scanner's laser beam). 1. High point density throughout the dataset. Each PTS file contains 8 columns: | Column ID | Description | |---------------|-------------------------------------------------| | 1 | X Cartesian coordinate | | 2 | Y Cartesian coordinate | | 3 | Z Cartesian Coordinate | | 4 | Red colour in RGB space in the range [0, 255] | | 5 | Green colour in RGB space in the range [0, 255] | | 6 | Blue colour in RGB space in the range [0, 255] | | 7 | Category code | | 8 | Instance ID |