Papers with Code 아카이브

1 Image, 2*2 Stitchi

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
12
평가표
1
보존된 평가 기록
1

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

Biwi 3D Audiovisual Corpus of Affective Communication - B3D(AC)^2

BIWI 3D corpus comprises a total of 1109 sentences uttered by 14 native English speakers (6 males and 8 females). A real time 3D scanner and a professional microphone were used to capture the facial movements and the speech of the speakers. The dense dynamic face scans were acquired at 25 frames per second and the RMS error in the 3D reconstruction is about 0.5 mm. In order to ease automatic speech segmentation, we carried out the recordings in a anechoic room, with walls covered by sound wave-absorbing materials. Each sentence was recorded twice: - First, the speaker read the sentence from text, with a neutral expression. - Then, the speaker watched a clip extracted from a feature film where the sentence is acted by professional actors and the context is highly emotional. After rating the emotions induced by the video, the speaker repeated the sentence.

BRACE

BRACE is a dataset for audio-conditioned dance motion synthesis challenging common assumptions for this task: - strong music-dance correlation - controlled motion data - simple poses and movements To address these issues: - We focus on breakdancing which features acrobatic moves, tangled postures and weaker dance-music correlation. - We adopt a hybrid labelling pipeline leveraging estimation models as well as manual annotations to obtain good quality keypoint sequences at a reduced cost. - Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses. - BRACE is also useful to fine-tune pose-estimation models thanks to its high quality keypoint annotations for complicated and uncommon poses.

ChronoMagic

ChronoMagic with 2265 metamorphic time-lapse videos, each accompanied by a detailed caption.

DREAM-dataset

The DREAM dataset is introduce by the paper "Camera-to-Robot Pose Estimation from a Single Image" (ICRA 2020). This dataset consists of synthetic images (both with and without domain randomlization) of three different robot manipulators (Franka Emika’s Panda, Kuka’s LBR iiwa 7 R800, and Rethink Robotics’ Baxter) , as well as real-world images of Franka Emika’s Panda taken from various RGBD cameras (XBox 360 Kinect (XK), RealSense (RS), and Azure Kinect (AK)). Each instance in the dataset contains an RGB image, keypoint 3D/2D coordinates , global camera-to-robot transformation and joint state configurations (from both revolute and prismatic joint) of the robot. Tasks like estimating robot pose (camera pose) from a single RGB image, camera-to-robot calibration can be conducted and evaluated in this dataset.

EgoPAT3D

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

HARPER

We introduce HARPER, a novel dataset for 3D body pose estimation and forecast in dyadic interactions between users and \spot, the quadruped robot manufactured by Boston Dynamics. The key-novelty is the focus on the robot's perspective, i.e., on the data captured by the robot's sensors. These make 3D body pose analysis challenging because being close to the ground captures humans only partially. The scenario underlying HARPER includes 15 actions, of which 10 involve physical contact between the robot and users. The Corpus contains not only the recordings of the built-in stereo cameras of Spot, but also those of a 6-camera OptiTrack system (all recordings are synchronized). This leads to ground-truth skeletal representations with a precision lower than a millimeter. In addition, the Corpus includes reproducible benchmarks on 3D Human Pose Estimation, Human Pose Forecasting, and Collision Prediction, all based on publicly available baseline approaches. This enables future HARPER users to rigorously compare their results with those we provide in this work. Source: [Download dataset] (https://github.com/intelligolabs/HARPER)

NERDS 360

We present a large-scale dataset for 3D urban scene understanding. Compared to existing datasets, our dataset consists of 75 outdoor urban scenes with diverse backgrounds, encompassing over 15,000 images. These scenes offer 360◦ hemispherical views, capturing diverse foreground objects illuminated under various lighting conditions. Additionally, our dataset encompasses scenes that are not limited to forward-driving views, addressing the limitations of previous datasets such as limited overlap and coverage between camera views. The closest pre-existing dataset for generalizable evaluation is DTU [2] (80 scenes) which comprises mostly indoor objects and does not provide multiple foreground objects or background scenes. We use the Parallel Domain synthetic data generation to render high-fidelity 360◦ scenes. We select 3 different maps i.e. SF 6thAndMission, SF GrantAndCalifornia and SF VanNessAveAndTurkSt and sample 75 different scenes across all 3 maps as our backgrounds (All 75 scenes across 3 maps are significantly different road scenes from each other, captured at different viewpoints in the city). We select 20 different cars in 50 different textures for training and randomly sample from 1 to 4 cars to render in a scene. We refer to this dataset as NeRDS 360: NeRF for Reconstruction, Decomposition and Scene Synthesis of 360◦ outdoor scenes. In total, we generate 15k renderings by sampling 200 cameras in a hemispherical dome at a fixed distance from the center of cars. We held out 5 scenes with 4 different cars and different backgrounds for testing, comprising 100 cameras distributed uniformly sampled in the upper hemisphere, different from the camera distributions used for training. We use the diverse validation camera distribution to test our approach’s ability to generalize to unseen viewpoints as well as unseen scenes during training. Our dataset and the corresponding task is extremely challenging due to occlusions, diversity of backgrounds, and rendered objects with various lightning and shadows. Our task entails reconstructing 360◦ hemispherical views of complete scenes using a handful of observations i.e. 1 to 5 as shown by red cameras whereas evaluating using all 100 hemispherical views, hence our task requires strong priors for novel view synthesis of outdoor scenes. Tasks our dataset support: - Generaliazable Novel view synthesis (Few shot evaluation) - Novel view synthesis (Overfitting evaluation) - 6D pose estimation - Object editing - Depth estimation - Semantic Segmentation - Instance Segmentation

PedX

PedX is a large-scale multi-modal collection of pedestrians at complex urban intersections. The dataset provides high-resolution stereo images and LiDAR data with manual 2D and automatic 3D annotations. The data was captured using two pairs of stereo cameras and four Velodyne LiDAR sensors. Source: pedx.io

Song Describer Dataset

The Song Describer Dataset (SDD) contains ~1.1k captions for 706 permissively licensed music recordings. It is designed for use in evaluation of models that address music-and-language (M&L) tasks such as music captioning, text-to-music generation and music-language retrieval.

TextAtlasEval

A Dense-text Image Benchmark to evaluate large generation model's ability on text generation.

AwA Pose

AwA Pose is a large scale animal keypoint dataset with ground truth annotations for keypoint detection of quadruped animals from images.

dMelodies

dMelodies is dataset of simple 2-bar melodies generated using 9 independent latent factors of variation where each data point represents a unique melody based on the following constraints: - Each melody will correspond to a unique scale (major, minor, blues, etc.). - Each melody plays the arpeggios using the standard I-IV-V-I cadence chord pattern. - Bar 1 plays the first 2 chords (6 notes), Bar 2 plays the second 2 chords (6 notes). - Each played note is an 8th note. Source: https://github.com/ashispati/dmelodies_dataset

FakeMusicCaps

The FakeMusicCaps dataset contains total of 27605 10 seconds music tracks corresponding to almost 77 hours, generated using 5 different Text-To-Music (TTM) models. It is designed to be used as a starting dataset for the training and/or evaluation of models for the detection and attribution of synthetic music generated via TTM models. If you use this dataset, please cite our paper: Comanducci, L., Bestagini, P., and Tubaro, S., “FakeMusicCaps: a Dataset for Detection and Attribution of Synthetic Music Generated via Text-to-Music Models”, Art. no. arXiv:2409.10684, 2024. doi:10.48550/arXiv.2409.10684.

FewSOL

The Few-Shot Object Learning (FewSOL) dataset can be used for object recognition with a few images per object. It contains 336 real-world objects with 9 RGB-D images per object from different views. Object segmentation masks, object poses and object attributes are provided. In addition, synthetic images generated using 330 3D object models are used to augment the dataset. FewSOL dataset can be used to study a set of few-shot object recognition problems such as classification, detection and segmentation, shape reconstruction, pose estimation, keypoint correspondences and attribute recognition. Motivation: If robots can recognize objects from a few exemplar images, it is possible to scale up the number of objects a robot can recognize because collecting a few images per object is a much easier process compared to building a 3D model of an object. In addition, models trained in the meta-learning setting can generalize to new objects without re-training.

Fraunhofer IPA Bin-Picking

The Fraunhofer IPA Bin-Picking dataset is a large-scale dataset comprising both simulated and real-world scenes for various objects (potentially having symmetries) and is fully annotated with 6D poses. A pyhsics simulation is used to create scenes of many parts in bulk by dropping objects in a random position and orientation above a bin. Additionally, this dataset extends the Siléane dataset by providing more samples. This allows to e.g. train deep neural networks and benchmark the performance on the public Siléane dataset Source: https://www.bin-picking.ai/en/dataset.html Image Source: https://arxiv.org/abs/1912.12125

HPD

These images were generated using Blender and IEE-Simulator with different head-poses, where the images are labelled according to nine classes (straight, turned bottom-left, turned left, turned top-left, turned bottom-right, turned right, turned top-right, reclined, looking up). The dataset contains 16,013 training images and 2,825 testing images, in addition to 4,700 images for improvements.

ImgEdit

ImgEdit is a large-scale, high-quality image-editing dataset comprising 1.2 million carefully curated edit pairs, which contain both novel and complex single-turn edits, as well as challenging multi-turn tasks.

MERL-RAV

The MERL-RAV (MERL Reannotation of AFLW with Visibility) Dataset contains over 19,000 face images in a full range of head poses. Each face is manually labeled with the ground-truth locations of 68 landmarks, with the additional information of whether each landmark is unoccluded, self-occluded (due to extreme head poses), or externally occluded. The images were annotated by professional labelers, supervised by researchers at Mitsubishi Electric Research Laboratories (MERL).

MPV

Consists of 37,723/14,360 person/clothes images, with the resolution of 256x192. Each person has different poses. We split them into the train/test set 52,236/10,544 three-tuples, respectively. You can download the dataset at MPV(Google Drive)

PASTEL

PASTEL is a parallelly annotated stylistic language dataset. The dataset consists of ~41K parallel sentences and 8.3K parallel stories annotated across different personas. Source: PASTEL

RealMCVSR

Our RealMCVSR dataset provides real-world HD video triplets concurrently recorded by Apple iPhone 12 Pro Max equipped with triple cameras having fixed focal lengths: ultra-wide (30mm), wide-angle (59mm), and telephoto (147mm). To concurrently record video triplets, we built an iOS app that provides full control over exposure parameters (i.e., shutter speed and ISO) of the cameras. For recording each scene, we set the cameras in the auto-exposure mode, where the shutter speeds of the three cameras are synced to avoid varying motion blur across a video triplet. ISOs are adjusted accordingly for each camera to pick up the same exposure. Each video is saved in the MOV format using HEVC/H.265 encoding with the HD resolution (1080 x 1920). The dataset contains triplets of 161 video clips with 23,107 frames in total. The video triplets are split into training, validation, and testing sets, each of which has 137, 8, and 16 triplets with 19,426, 1,141, and 2,540 frames, respectively.

Relative Human

Relative Human (RH) contains multi-person in-the-wild RGB images with rich human annotations, including: Depth layers: relative depth relationship/ordering between all people in the image. Age group classfication: adults, teenagers, kids, babies. Others: Genders, Bounding box, 2D pose.

SportsPose

Accurate 3D human pose estimation is essential for sports analytics, coaching, and injury prevention. However, existing datasets for monocular pose estimation do not adequately capture the challenging and dynamic nature of sports movements. In response, we introduce SportsPose, a large-scale 3D human pose dataset consisting of highly dynamic sports movements. With more than 176,000 3D poses from 24 different subjects performing 5 different sports activities, SportsPose provides a diverse and comprehensive set of 3D poses that reflect the complex and dynamic nature of sports movements. Contrary to other markerless datasets we have quantitatively evaluated the precision of SportsPose by comparing our poses with a commercial marker-based system and achieve a mean error of 34.5 mm across all evaluation sequences. This is comparable to the error reported on the commonly used 3DPW dataset. We further introduce a new metric, local movement, which describes the movement of the wrist and ankle joints in relation to the body. With this, we show that SportsPose contains more movement than the Human3.6M and 3DPW datasets in these extremum joints, indicating that our movements are more dynamic. The dataset with accompanying code can be downloaded from our website. We hope that SportsPose will allow researchers and practitioners to develop and evaluate more effective models for the analysis of sports performance and injury prevention. With its realistic and diverse dataset, SportsPose provides a valuable resource for advancing the state-of-the-art in pose estimation in sports

3D-POP

The dataset is designed specifically to solve a range of computer vision problems (2D-3D tracking, posture) faced by biologists while designing behavior studies with animals. Typically, datasets for animal-specific vision tasks are created using open-source video material. This might be effective for an initial start, but these methods are not deployment ready for the behavior community. Therefore, we designed a semi-automated method for biologists to create well-curated datasets at a large scale for the ML and Vision community. 3D-POP is the first dataset with 3D ground truth for multi-animal, multi-view tracking problems. Highlight: The dataset is captured with the intention of using it for various vision problems and with different levels of complexity (no of cameras, no of individuals) Video explanation: Link to YouTube video Video teaser: Link to YouTube video ## Dataset Features: ### Marker-based videos: - 6 hours+ of annotations of 18 individuals (groups of 1, 2, 5, 10). - Bounding box - Trajectories (2D and 3D) - Posture (2D and 3D) with 9 key points - Identities - Total of 57 sequences (4K) with 4 views. - Dataset customization* (Users can modify the dataset and add key points to the dataset) ### Markerless: - 1Hr+ videos of 18 individuals in groups of 1, 2, 5, 11. The birds have no markers. This data is provided as test cases and unsupervised approaches. ## Problems: ### 2D domain: - Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview. - Tracking with single - multiview ### 3D domain: - Position, Posture of birds (different group sizes n = 1, 2, 5, 10) with Single/Multiview. - Tracking with single - multiview ### Fine-grained recognition: - Identity tracking with ground truth. ### Unsupervised learning: - 2D or 3D posture problems ## Idea: The dataset is created with a motion capture system, using the 6-DOF tracking ability. Assumptions are that head and body act as rigid bodies when birds walk and forage (proved with experiment). Therefore, we get the 3D position of key points by tracking head/body orientation.

A Billion Ways to Grasp

Robot grasping is often formulated as a learning problem. With the increasing speed and quality of physics simulations, generating large-scale grasping data sets that feed learning algorithms is becoming more and more popular. An often overlooked question is how to generate the grasps that make up these data sets. In this paper, we review, classify, and compare different grasp sampling strategies. Our evaluation is based on a fine-grained discretization of SE(3) and uses physics-based simulation to evaluate the quality and robustness of the corresponding parallel-jaw grasps. Specifically, we consider more than 1 billion grasps for each of the 21 objects from the YCB data set. This dense data set lets us evaluate existing sampling schemes w.r.t. their bias and efficiency. Our experiments show that some popular sampling schemes contain significant bias and do not cover all possible ways an object can be grasped.

ACL Title and Abstract Dataset

This dataset gathers 10,874 title and abstract pairs from the ACL Anthology Network (until 2016). The structure of the data is as follows: - title - abstract - \newline This dataset is used in our published paper: Paper Abstract Writing through Editing Mechanism ## Citation @inproceedings{wang-etal-2018-paper, title = "Paper Abstract Writing through Editing Mechanism", author = "Wang, Qingyun and Zhou, Zhihao and Huang, Lifu and Whitehead, Spencer and Zhang, Boliang and Ji, Heng and Knight, Kevin", booktitle = "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)", month = jul, year = "2018", address = "Melbourne, Australia", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/P18-2042", doi = "10.18653/v1/P18-2042", pages = "260--265", abstract = "We present a paper abstract writing system based on an attentive neural sequence-to-sequence model that can take a title as input and automatically generate an abstract. We design a novel Writing-editing Network that can attend to both the title and the previously generated abstract drafts and then iteratively revise and polish the abstract. With two series of Turing tests, where the human judges are asked to distinguish the system-generated abstracts from human-written ones, our system passes Turing tests by junior domain experts at a rate up to 30{\%} and by non-expert at a rate up to 80{\%}.", }

Composable activities dataset

The Composable activities dataset consists of 693 videos that contain activities in 16 classes performed by 14 actors. Each activity is composed of 3 to 11 atomic actions. RGB-D data for each sequence is captured using a Microsoft Kinect sensor and estimate position of relevant body joints. The dataset provides annotations of the activity for each video and the actions for each of the four human parts (left/right arm and leg) for each frame in every video. Source: Discriminative Hierarchical Modeling of Spatio-Temporally Composable Human Activities Image Source: https://ialillo.sitios.ing.uc.cl/ActionsCVPR2014/

DAD-3DHeads

DAD-3DHeads dataset consists of 44,898 images collected from various sources (37,840 in the training set, 4,312 in the validation set, and 2,746 in the test set). The dataset is well-balanced over a wide range of poses, face expressions, and occlusions. The attribute labels are a valuable signal for subgroup analysis and for generalisation to in-the-wild deployment conditions.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.