Papers with Code 아카이브

1 Image, 2*2 Stitchi

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
12
평가표
1
보존된 평가 기록
1

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

Panoptic

CMU Panoptic is a large scale dataset providing 3D pose annotations (1.5 millions) for multiple people engaging social activities. It contains 65 videos (5.5 hours) with multi-view annotations, but only 17 of them are in multi-person scenario and have the camera parameters. Massively Multiview System * 480 VGA camera views * 30+ HD views * 10 RGB-D sensors * Hardware-based sync * Calibration * Interesting Scenes with Labels Multiple people * Socially interacting groups * 3D body pose * 3D facial landmarks * Transcripts + speaker ID Hardware setup * 480 VGA cameras, 640 x 480 resolution, 25 fps, synchronized among themselves using a hardware clock * 31 HD cameras, 1920 x 1080 resolution, 30 fps, synchronized among themselves using a hardware clock, timing aligned with VGA cameras * 10 Kinect Ⅱ Sensors. 1920 x 1080 (RGB), 512 x 424 (depth), 30 fps, timing aligned among themselves and other sensors 5 DLP Projectors. synchronized with HD cameras Source: Single-Stage Multi-Person Pose Machines Image Source: http://domedb.perception.cs.cmu.edu/

FreiHAND

FreiHAND is a 3D hand pose dataset which records different hand actions performed by 32 people. For each hand image, MANO-based 3D hand pose annotations are provided. It currently contains 32,560 unique training samples and 3960 unique samples for evaluation. The training samples are recorded with a green screen background allowing for background removal. In addition, it applies three different post processing strategies to training samples for data augmentation. However, these post processing strategies are not applied to evaluation samples. Source: Knowledge as Priors: Cross-Modal Knowledge Generalizationfor Datasets without Superior Knowledge Image Source: https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html

NCLT

The NCLT dataset is a large scale, long-term autonomy dataset for robotics research collected on the University of Michigan’s North Campus. The dataset consists of omnidirectional imagery, 3D lidar, planar lidar, GPS, and proprioceptive sensors for odometry collected using a Segway robot. The dataset was collected to facilitate research focusing on long-term autonomous operation in changing environments. The dataset is comprised of 27 sessions spaced approximately biweekly over the course of 15 months. The sessions repeatedly explore the campus, both indoors and outdoors, on varying trajectories, and at different times of the day across all four seasons. This allows the dataset to capture many challenging elements including: moving obstacles (e.g., pedestrians, bicyclists, and cars), changing lighting, varying viewpoint, seasonal and weather changes (e.g., falling leaves and snow), and long-term structural changes caused by construction projects. Source: http://robots.engin.umich.edu/nclt/nclt.pdf Image Source: http://robots.engin.umich.edu/nclt/

MAESTRO

The MAESTRO dataset contains over 200 hours of paired audio and MIDI recordings from ten years of International Piano-e-Competition. The MIDI data includes key strike velocities and sustain/sostenuto/una corda pedal positions. Audio and MIDI files are aligned with ∼3 ms accuracy and sliced to individual musical pieces, which are annotated with composer, title, and year of performance. Uncompressed audio is of CD quality or higher (44.1–48 kHz 16-bit PCM stereo). Source: https://magenta.tensorflow.org/datasets/maestro Image Source: https://www.researchgate.net/figure/Results-generated-with-the-MAESTRO-dataset_fig3_333392458

COFW

The Caltech Occluded Faces in the Wild (COFW) dataset is designed to present faces in real-world conditions. Faces show large variations in shape and occlusions due to differences in pose, expression, use of accessories such as sunglasses and hats and interactions with objects (e.g. food, hands, microphones, etc.). All images were hand annotated using the same 29 landmarks as in LFPW. Both the landmark positions as well as their occluded/unoccluded state were annotated. The faces are occluded to different degrees, with large variations in the type of occlusions encountered. COFW has an average occlusion of over 23. Source: http://www.vision.caltech.edu/xpburgos/ICCV13/#dataset Image Source: http://www.vision.caltech.edu/xpburgos/ICCV13/#dataset

Hypersim

For many fundamental scene understanding tasks, it is difficult or impossible to obtain per-pixel ground truth labels from real images. Hypersim is a photorealistic synthetic dataset for holistic indoor scene understanding. It contains 77,400 images of 461 indoor scenes with detailed per-pixel labels and corresponding ground truth geometry. Source: https://github.com/apple/ml-hypersim Image Source: https://github.com/apple/ml-hypersim

GYAFC

Grammarly’s Yahoo Answers Formality Corpus (GYAFC) is the largest dataset for any style containing a total of 110K informal / formal sentence pairs. Yahoo Answers is a question answering forum, contains a large number of informal sentences and allows redistribution of data. The authors used the Yahoo Answers L6 corpus to create the GYAFC dataset of informal and formal sentence pairs. In order to ensure a uniform distribution of data, they removed sentences that are questions, contain URLs, and are shorter than 5 words or longer than 25. After these preprocessing steps, 40 million sentences remain. The Yahoo Answers corpus consists of several different domains like Business, Entertainment & Music, Travel, Food, etc. Pavlick and Tetreault formality classifier (PT16) shows that the formality level varies significantly across different genres. In order to control for this variation, the authors work with two specific domains that contain the most informal sentences and show results on training and testing within those categories. The authors use the formality classifier from PT16 to identify informal sentences and train this classifier on the Answers genre of the PT16 corpus which consists of nearly 5,000 randomly selected sentences from Yahoo Answers manually annotated on a scale of -3 (very informal) to 3 (very formal). They find that the domains of Entertainment & Music and Family & Relationships contain the most informal sentences and create the GYAFC dataset using these domains. Source: Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer

PoseTrack

The PoseTrack dataset is a large-scale benchmark for multi-person pose estimation and tracking in videos. It requires not only pose estimation in single frames, but also temporal tracking across frames. It contains 514 videos including 66,374 frames in total, split into 300, 50 and 208 videos for training, validation and test set respectively. For training videos, 30 frames from the center are annotated. For validation and test videos, besides 30 frames from the center, every fourth frame is also annotated for evaluating long range articulated tracking. The annotations include 15 body keypoints location, a unique person id and a head bounding box for each person instance. Source: Simple Baselines for Human Pose Estimation and Tracking Image Source: https://posetrack.net/

Fashion IQ

Fashion IQ support and advance research on interactive fashion image retrieval. Fashion IQ is the first fashion dataset to provide human-generated captions that distinguish similar pairs of garment images together with side-information consisting of real-world product descriptions and derived visual attribute labels for these images. Source: Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback

GenEval

Recent breakthroughs in diffusion models, multimodal pretraining, and efficient finetuning have led to an explosion of text-to-image generative models. Given human evaluation is expensive and difficult to scale, automated methods are critical for evaluating the increasingly large number of new models. However, most current automated evaluation metrics like FID or CLIPScore only offer a holistic measure of image quality or image-text alignment, and are unsuited for fine-grained or instance-level analysis. In this paper, we introduce GenEval, an object-focused framework to evaluate compositional image properties such as object co-occurrence, position, count, and color. We show that current object detection models can be leveraged to evaluate text-to-image models on a variety of generation tasks with strong human agreement, and that other discriminative vision models can be linked to this pipeline to further verify properties like object color. We then evaluate several open-source text-to-image models and analyze their relative generative capabilities on our benchmark. We find that recent models demonstrate significant improvement on these tasks, though they are still lacking in complex capabilities such as spatial relations and attribute binding. Finally, we demonstrate how GenEval might be used to help discover existing failure modes, in order to inform development of the next generation of text-to-image models. Our code to run the GenEval framework is publicly available at this https URL.

Real Blur Dataset

The dataset consists of 4,738 pairs of images of 232 different scenes including reference pairs. All images were captured both in the camera raw and JPEG formats, hence generating two datasets: RealBlur-R from the raw images, and RealBlur-J from the JPEG images. Each training set consists of 3,758 image pairs, while each test set consists of 980 image pairs. The deblurring result is first aligned to its ground truth sharp image using a homography estimated by the enhanced correlation coefficients method, and PSNR or SSIM is computed in sRGB color space.

HIDE

Consists of 8,422 blurry and sharp image pairs with 65,784 densely annotated FG human bounding boxes. Source: Human-Aware Motion Deblurring

DexYCB

DexYCB is a dataset for capturing hand grasping of objects. It can be used three relevant tasks: 2D object and keypoint detection, 6D object pose estimation, and 3D hand pose estimation. The dataset was built using 20 objects from the YCB-Video dataset, and consists of multiple trials from 10 subjects. For each trial, there is a target object with 2 to 4 other objects placed on a table. The subject is asked to start from a relaxed pose, pick up the target object, and hold it in the air. Some subjects were asked to pretend to hand over the object to someone across from them. Each action is recorded for 3 seconds, repeating the trial 5 times for each target object, each time with a random set of accompanied objects and placement. In total there are 100 trials per subject, and 1,000 trials in total for all subjects. Source: DexYCB: A Benchmark for Capturing Hand Grasping of Objects

T-LESS

T-LESS is a dataset for estimating the 6D pose, i.e. translation and rotation, of texture-less rigid objects. The dataset features thirty industry-relevant objects with no significant texture and no discriminative color or reflectance properties. The objects exhibit symmetries and mutual similarities in shape and/or size. Compared to other datasets, a unique property is that some of the objects are parts of others. The dataset includes training and test images that were captured with three synchronized sensors, specifically a structured-light and a time-of-flight RGB-D sensor and a high-resolution RGB camera. There are approximately 39K training and 10K test images from each sensor. Additionally, two types of 3D models are provided for each object, i.e. a manually created CAD model and a semi-automatically reconstructed one. Training images depict individual objects against a black background. Test images originate from twenty test scenes having varying complexity, which increases from simple scenes with several isolated objects to very challenging ones with multiple instances of several objects and with a high amount of clutter and occlusion. The images were captured from a systematically sampled view sphere around the object/scene, and are annotated with accurate ground truth 6D poses of all modeled objects. Source: http://cmp.felk.cvut.cz/t-less/ Image Source: http://cmp.felk.cvut.cz/t-less/

Aachen Day-Night

Aachen Day-Night is a dataset designed for benchmarking 6DOF outdoor visual localization in changing conditions. It focuses on localizing high-quality night-time images against a day-time 3D model. There are 14,607 images with changing conditions of weather, season and day-night cycles. Source: Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions

VITON

VITON was a dataset for virtual try-on of clothing items. It consisted of 16,253 pairs of images of a person and a clothing item . The authors have removed the dataset and it is no longer publicly available due to copyright issues.

PartNet-Mobility

Dataset produced for the SAPIEN simulation environment. From the website: "PartNet-Mobility dataset is a collection of 2K articulated objects with motion annotations and rendernig material. The dataset powers research for generalizable computer vision and manipulation. The dataset is a continuation of ShapeNet and PartNet. "

MusicCaps

MusicCaps is a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts. For each 10-second music clip, MusicCaps provides: 1) A free-text caption consisting of four sentences on average, describing the music and 2) A list of music aspects, describing genre, mood, tempo, singer voices, instrumentation, dissonances, rhythm, etc. Source:MusicLM: Generating Music From Text

VITON-HD

VITON-HD dataset is a dataset for high-resolution (i.e., 1024x768) virtual try-on of clothing items. Specifically, it consists of 13,679 frontal-view woman and top clothing image pairs.

DrawBench

DrawBench is a comprehensive and challenging benchmark for text-to-image models, introduced by the Imagen research team. Let me provide you with more details: 1. Purpose and Context: - DrawBench serves as an evaluation benchmark specifically designed to assess the performance of text-to-image models. - It allows researchers and practitioners to compare different methods and understand their strengths and weaknesses in generating images from textual descriptions. 2. Imagen: Text-to-Image Diffusion Models: - Imagen is a state-of-the-art text-to-image diffusion model developed by the Google Research Brain Team. - It combines the power of large transformer language models (such as T5) for understanding text with the strength of diffusion models for high-fidelity image generation. - Key Discovery: Imagen demonstrates that generic large language models pretrained on text-only corpora are remarkably effective at encoding text for image synthesis. - Photorealism and Language Understanding: Imagen achieves an unprecedented degree of photorealism and a deep level of language understanding. - FID Score: It achieves a new state-of-the-art FID (Fréchet Inception Distance) score of 7.27 on the COCO dataset, without ever being trained on COCO. - Human Raters' Perception: Human raters find Imagen samples to be on par with the COCO data itself in terms of image-text alignment. 3. DrawBench: A Comprehensive Benchmark: - DrawBench provides a rigorous evaluation framework for text-to-image models. - Researchers can compare Imagen with other recent methods, including VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2. - Human raters prefer Imagen over other models in side-by-side comparisons, considering both sample quality and image-text alignment. 4. Examples from the Imagen Family: - Imagen generates diverse and imaginative images based on textual prompts. Here are some examples: - A strawberry mug filled with white sesame seeds, floating in a dark chocolate sea. - A brain riding a rocketship heading towards the moon. - A dragon fruit wearing a karate belt in the snow. - A small cactus wearing a straw hat and neon sunglasses in the Sahara desert. - A photo of a Corgi dog riding a bike in Times Square, wearing sunglasses and a beach hat. - Teddy bears swimming at the Olympics 400m Butterfly event. - Sprouts in the shape of the text 'Imagen' coming out of a fairytale book. - A transparent sculpture of a duck made out of glass, in front of a painting of a landscape. - A single beam of light entering the room from the ceiling, illuminating an easel with a Rembrandt painting of a raccoon. 5. Technical Details: - Imagen uses a large frozen T5-XXL encoder to encode input text into embeddings. - The combination of language understanding and diffusion-based image generation results in high-quality, contextually relevant images. Source: Conversation with Bing, 3/18/2024 (1) Imagen: Text-to-Image Diffusion Models. https://imagen.research.google/. (2) Evaluating Diffusion Models - Hugging Face. https://huggingface.co/docs/diffusers/conceptual/evaluation. (3) shunk031/DrawBench · Datasets at Hugging Face. https://huggingface.co/datasets/shunk031/DrawBench. (4) sayakpaul/drawbench · Datasets at Hugging Face. https://huggingface.co/datasets/sayakpaul/drawbench.

MPIIGaze

MPIIGaze is a dataset for appearance-based gaze estimation in the wild. It contains 213,659 images collected from 15 participants during natural everyday laptop use over more than three months. It has a large variability in appearance and illumination. Source: https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/gaze-based-human-computer-interaction/appearance-based-gaze-estimation-in-the-wild Image Source: https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/gaze-based-human-computer-interaction/appearance-based-gaze-estimation-in-the-wild

RHD

Rendered Hand Pose (RHD) is a dataset for hand pose estimation. It provides segmentation maps with 33 classes: three for each finger, palm, person, and background. The 3D kinematic model of the hand provides 21 keypoints per hand: 4 keypoints per finger and one keypoint close to the wrist.

GRAB

GRAB is a dataset of full-body motions interacting and grasping 3D objects. It contains accurate finger and facial motions as well as the contact between the objects and body. It contains 5 male and 5 female participants and 4 different motion intents. The GRAB dataset also contains binary contact maps between the body and objects. Source: https://github.com/otaheri/GRAB Image Source: https://github.com/otaheri/GRAB

AGORA

AGORA is a synthetic human dataset with high realism and accurate ground truth. It consists of around 14K training and 3K test images by rendering between 5 and 15 people per image using either image-based lighting or rendered 3D environments, taking care to make the images physically plausible and photoreal. In total, AGORA contains 173K individual person crops. AGORA provides (1) SMPL/SMPL-X parameters and (2) segmentation masks for each subject in images.

InLoc

InLoc is a dataset with reference 6DoF poses for large-scale indoor localization. Query photographs are captured by mobile phones at a different time than the reference 3D map, thus presenting a realistic indoor localization scenario. Source: InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

REAL275

REAL275 is a benchmark for category-level pose estimation. It contains 4300 training frames, 950 validation and 2750 for testing across 18 different real scenes.

T2I-CompBench

T2I-CompBench is a comprehensive benchmark for open-world compositional text-to-image generation, consisting of 6,000 compositional textual prompts from 3 categories (attribute binding, object relationships, and complex compositions) and 6 sub-categories (color binding, shape binding, texture binding, spatial relationships, non-spatial relationships, and complex compositions).

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.