태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
Visual Navigation is the problem of navigating an agent, e.g. a mobile robot, in an environment using camera input only. The agent is given a target image (an image it will see from the target position), and its goal is to move from its current position to the target by applying a sequence of actions, based on the camera observations only.
Source: Vision-based Navigation Using Deep Reinforcement Learning
출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.
The Replica Dataset is a dataset of high quality reconstructions of a variety of indoor spaces. Each reconstruction has clean dense geometry, high resolution and high dynamic range textures, glass and mirror surface information, planar segmentation as well as semantic class and instance segmentation. Source: The Replica Dataset: A Digital Replica of Indoor Spaces
AI2-Thor is an interactive environment for embodied AI. It contains four types of scenes, including kitchen, living room, bedroom and bathroom, and each scene includes 30 rooms, where each room is unique in terms of furniture placement and item types. There are over 2000 unique objects for AI agents to interact with. Source: Learning Object Relation Graph andTentative Policy for Visual Navigation Image Source: https://ai2thor.allenai.org/
SUNCG is a large-scale dataset of synthetic 3D scenes with dense volumetric annotations. The dataset is currently not available. Source: https://sscnet.cs.princeton.edu/ Image Source: https://sscnet.cs.princeton.edu/
R2R is a dataset for visually-grounded natural language navigation in real buildings. The dataset requires autonomous agents to follow human-generated navigation instructions in previously unseen buildings, as illustrated in the demo above. For training, each instruction is associated with a Matterport3D Simulator trajectory. 22k instructions are available, with an average length of 29 words. There is a test evaluation server for this dataset available at EvalAI. Source: Natural language interaction with robots
The 2D-3D-S dataset provides a variety of mutually registered modalities from 2D, 2.5D and 3D domains, with instance-level semantic and geometric annotations. It covers over 6,000 m2 collected in 6 large-scale indoor areas that originate from 3 different buildings. It contains over 70,000 RGB images, along with the corresponding depths, surface normals, semantic annotations, global XYZ images (all in forms of both regular and 360° equirectangular images) as well as camera information. It also includes registered raw and semantically annotated 3D meshes and point clouds. The dataset enables development of joint and cross-modal learning models and potentially unsupervised approaches utilizing the regularities present in large-scale indoor spaces. Source: https://github.com/alexsax/2D-3D-Semantics Image Source: https://github.com/alexsax/2D-3D-Semantics
The HELP dataset is an automatically created natural language inference (NLI) dataset that embodies the combination of lexical and logical inferences focusing on monotonicity (i.e., phrase replacement-based reasoning). The HELP (Ver.1.0) has 36K inference pairs consisting of upward monotone, downward monotone, non-monotone, conjunction, and disjunction. Source: HELP
AVD focuses on simulating robotic vision tasks in everyday indoor environments using real imagery. The dataset includes 20,000+ RGB-D images and 50,000+ 2D bounding boxes of object instances densely captured in 9 unique scenes. Source: A Dataset for Developing and Benchmarking Active Vision
MINOS is a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments. MINOS leverages large datasets of complex 3D environments and supports flexible configuration of multimodal sensor suites.
The Habitat-Matterport 3D Semantics Dataset (HM3DSem) is the largest-ever dataset of 3D real-world and indoor spaces with densely annotated semantics that is available to the academic community. HM3DSem v0.2 consists of 142,646 object instance annotations across 216 3D-spaces from HM3D and 3,100 rooms within those spaces. The HM3D scenes are annotated with the 142,646 raw object names, which are mapped to 40 Matterport categories. On average, each scene in HM3DSem v0.2 consists of 661 objects from 106 categories. This dataset is the result of 14,200+ hours of human effort for annotation and verification by 20+ annotators. HM3DSem v0.2 is free and available here for academic, non-commercial research. Researchers can use it with FAIR’s Habitat simulator to train embodied agents, such as home robots and AI assistants, at scale for semantic navigation tasks. HM3DSem v0.1 was also the basis of the recently concluded Habitat 2022 ObjectNav challenge. Please see our arxiv report for more details.
A rich, extensible and efficient environment that contains 45,622 human-designed 3D scenes of visually realistic houses, ranging from single-room studios to multi-storied houses, equipped with a diverse set of fully labeled 3D objects, textures and scene layouts, based on the SUNCG dataset (Song et.al.) Source: Building Generalizable Agents with a Realistic and Rich 3D Environment
Talk The Walk is a large-scale dialogue dataset grounded in action and perception. The task involves two agents (a “guide” and a “tourist”) that communicate via natural language in order to achieve a common goal: having the tourist navigate to a given target location.
IQUAD is a dataset for Visual Question Answering in interactive environments. It is built upon AI2-THOR, a simulated photo-realistic environment of configurable indoor scenes with interactive object. IQUAD V1 has 75,000 questions, each paired with a unique scene configuration. Source: IQA: Visual Question Answering in Interactive Environments
MineRLis an imitation learning dataset with over 60 million frames of recorded human player data. The dataset includes a set of tasks which highlights many of the hardest problems in modern-day Reinforcement Learning: sparse rewards and hierarchical policies. Source: MineRL
BASEPROD provides comprehensive rover sensor data collected over a 1.7 km traverse, accompanied by high-resolution 2D and 3D drone maps of the terrain. The dataset also includes laser-induced breakdown spectroscopy (LIBS) measurements from key sampling sites along the rover's path, as well as weather station data to contextualize environmental conditions. The rover is a half-scale model of the ExoMars rover, featuring a sophisticated six-wheel triple-bogie suspension system. Each wheel assembly includes three actuators— one for wheel deployment (see e.g., wheel-walking), one for steering, and one for driving — and is equipped with a force-torque sensor. The sensor suite also comprises an RGB-D camera, a thermal camera, and a stereo camera mounted on a mast with pan-tilt capability. Additionally, the dataset contains IMU, GNSS, and gyroscope measurements, facilitating comprehensive analysis. Potential Applications: Research in navigation, localization, mapping, odometry, and terrain classification or segmentation tasks. It provides a rich resource for developing and testing algorithms in planetary exploration conditions.
MIDGARD is an open-source simulator for autonomous robot navigation in outdoor unstructured environments. It is designed to enable the training of autonomous agents (e.g., unmanned ground vehicles) in photorealistic 3D environments, and support the generalization skills of learning-based agents thanks to the variability in training scenarios.
Talk2Nav is a large-scale dataset with verbal navigation instructions. Source: Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory Image Source: https://www.trace.ethz.ch/publications/2019/talk2nav/index.html
A dataset for Image-Goal Navigation in Habitat based on Gibson scenes.
This is the supporting dataset for the ECCV 2024 paper "MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain". It contains 5,067 cropped images of craters on the surface of the Moon, generated in the Blender software using publicly available NASA mission data. Also included are 2,161 replicate orbital navigation frames from real-world Lunar Reconnaissance Orbiter (LRO) spacecraft poses with ground-truth bounding box annotations. Luna-1 supports the evaluation of learning-based vision systems for Terrain Relative Navigation (TRN) spacecraft applications.
Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.