Occlusion Handling

Separated COCO is automatically generated subsets of COCO val dataset, collecting separated objects for a large variety of categories in real images in a scalable manner, where target object segmentation mask is separated into distinct regions by the occluder.

The dataset is designed specifically to solve a range of computer vision problems (2D-3D tracking, posture) faced by biologists while designing behavior studies with animals. Typically, datasets for animal-specific vision tasks are created using open-source video material. This might be effective for an initial start, but these methods are not deployment ready for the behavior community. Therefore, we designed a semi-automated method for biologists to create well-curated datasets at a large scale for the ML and Vision community. 3D-POP is the first dataset with 3D ground truth for multi-animal, multi-view tracking problems. Highlight: The dataset is captured with the intention of using it for various vision problems and with different levels of complexity (no of cameras, no of individuals) Video explanation: Link to YouTube video Video teaser: Link to YouTube video ## Dataset Features: ### Marker-based videos: - 6 hours+ of annotations of 18 individuals (groups of 1, 2, 5, 10). - Bounding box - Trajectories (2D and 3D) - Posture (2D and 3D) with 9 key points - Identities - Total of 57 sequences (4K) with 4 views. - Dataset customization* (Users can modify the dataset and add key points to the dataset) ### Markerless: - 1Hr+ videos of 18 individuals in groups of 1, 2, 5, 11. The birds have no markers. This data is provided as test cases and unsupervised approaches. ## Problems: ### 2D domain: - Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview. - Tracking with single - multiview ### 3D domain: - Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview. - Tracking with single - multiview ### Fine-grained recognition: - Identity tracking with ground truth. ### Unsupervised learning: - 2D or 3D posture problems ## Idea: The dataset is created with a motion capture system, using the 6-DOF tracking ability. Assumptions are that head and body act as rigid bodies when birds walk and forage (proved with experiment). Therefore, we get the 3D position of key points by tracking head/body orientation.

Occluded COCO is automatically generated subset of COCO val dataset, collecting partially occluded objects for a large variety of categories in real images in a scalable manner, where target object is partially occluded but the segmentation mask is connected.

A occluded version of the LFW dataset for occluded face recognition verification. Uses structured occlusions generated to seem more realistic.

Overview The IITKGP_Fence dataset is designed for tasks related to fence-like occlusion detection, defocus blur, depth mapping, and object segmentation. The captured data vaies in scene composition, background defocus, and object occlusions. The dataset comprises both labeled and unlabeled data, as well as additional video and RGB-D data. The contains ground truth occlusion masks (GT) for the corresponding images. We created the ground truth occlusion labels in a semi-automatic way with user interaction. ## Key Dataset Features: - Fence Detection: Designed for detecting fences or fence-like structures that might occlude objects. - Defocus Blur: Also contains images and videos with blurred objects, likely to challenge detection and segmentation algorithms. - RGBD Data: Offers depth information alongside RGB images, which can be used for tasks like 3D reconstruction or occlusion handling. - Unlabeled and Labeled Data: Facilitates both supervised and unsupervised learning tasks. The Labeled <a href="https://huggingface.co/datasets/NeuroVizv0yaZ3R/IITKGP_Fence_dataset/tree/main/Labeled" style="color: darkorange;">folder</a> data provides ground truth occlusion masks, while the Unlabeled <a href="https://huggingface.co/datasets/NeuroVizv0yaZ3R/IITKGP_Fence_dataset/tree/main/Unlabeled" style="color: teal;">folder</a> data allows for further experimentation or self-supervised methods. ### Dataset Repository - GitHub Repository: <a href="https://github.com/Moushumi9medhi/Occlusion-Removal" style="color: magenta;">Occlusion-Removal</a> - Paper: Deep Generative Adversarial Network for Occlusion Removal from a Single Image - Authors: Sankaraganesh Jonna, Moushumi Medhi, Rajiv Ranjan Sahay ## Contact medhi.moushumi@iitkgp.ac.in