Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 27편

DeeperForensics-1.0

DeeperForensics-1.0 설명 보기

DeeperForensics-1.0 represents the largest face forgery detection dataset by far, with 60,000 videos constituted by a total of 17.6 million frames, 10 times larger than existing datasets of the same kind. The full dataset includes 48,475 source videos and 11,000 manipulated videos. The source videos are collected on 100 paid and consented actors from 26 countries, and the manipulated videos are generated by a newly proposed many-to-many end-to-end face swapping method, DF-VAE. 7 types of real-world perturbations at 5 intensity levels are employed to ensure a larger scale and higher diversity. Image Source: https://github.com/EndlessSora/DeeperForensics-1.0

관련 논문 27편

Evidence Inference

Evidence Inference 설명 보기

Evidence Inference is a corpus for this task comprising 10,000+ prompts coupled with full-text articles describing RCTs. Source: Inferring Which Medical Treatments Work from Reports of Clinical Trials

관련 논문 27편

FinanceBench

FinanceBench 설명 보기

Certainly! FinanceBench is a groundbreaking benchmark designed for evaluating the performance of large language models (LLMs) in the domain of financial question answering (QA). Here are the key details about FinanceBench: 1. What is FinanceBench? - FinanceBench is a first-of-its-kind test suite specifically tailored for assessing LLMs' capabilities in answering financial questions. - It focuses on open book financial QA and comprises a collection of 10,231 questions related to publicly traded companies. - Each question comes with corresponding answers and evidence strings. 2. Why FinanceBench Matters: - The questions in FinanceBench are ecologically valid, covering a diverse set of scenarios. - They are intentionally designed to be clear-cut and straightforward, serving as a minimum performance standard. - FinanceBench aims to evaluate how well LLMs handle financial queries, especially those related to publicly traded companies. 3. Model Evaluation: - Researchers tested 16 state-of-the-art model configurations, including GPT-4-Turbo, Llama2, and Claude2. - The evaluation involved a sample of 150 cases from FinanceBench, with manual review of answers (totaling 2,400). - Notably, existing LLMs have limitations for financial QA. For instance: - GPT-4-Turbo, when used with a retrieval system, incorrectly answered or refused to answer 81% of questions. - Augmentation techniques (such as longer context windows) improved performance but are unrealistic for enterprise settings due to increased latency and inability to handle larger financial documents. - All examined models exhibit weaknesses, such as hallucinations, which limit their suitability for enterprise use. 4. Availability: - The FinanceBench cases are available open-source for further exploration and research. ¹: Islam, P., Kannappan, A., Kiela, D., Qian, R., Scherrer, N., & Vidgen, B. (2023). FinanceBench: A New Benchmark for Financial Question Answering. arXiv preprint arXiv:2311.11944. ²: Link to the official paper ³: Papers with Code - FinanceBench Source: Conversation with Bing, 3/16/2024 (1) Papers with Code - FinanceBench: A New Benchmark for Financial Question .... https://paperswithcode.com/paper/financebench-a-new-benchmark-for-financial. (2) FinanceBench: A New Benchmark for Financial Question Answering. https://arxiv.org/abs/2311.11944. (3) Papers with Code - Paper tables with annotated results for FinanceBench .... https://paperswithcode.com/paper/financebench-a-new-benchmark-for-financial/review/.

관련 논문 27편

Gaofen Image Dataset

GID
GID 설명 보기

Gaofen Image Dataset (GID) is a large-scale land-cover dataset constructed with Gaofen-2 (GF-2) satellite images. This dataset has superiorities over the existing land-cover dataset because of its large coverage, wide distribution, and high spatial resolution. It contains 150 GF-2 images annotated at the pixel level for 5 categories: built-up, farmland, forest, meadow, and water.

관련 논문 27편

HTRU2

HTRU2 설명 보기

Pulsar candidates collected during the HTRU survey. Pulsars are a type of star, of considerable scientific interest. Candidates must be classified in to pulsar and non-pulsar classes to aid discovery.

관련 논문 27편

ImageNet-1k vs Textures

ImageNet-1k vs Textures 설명 보기

A benchmark dataset for out-of-distribution detection. ImageNet-1k is in-distribution, while Textures is out-of-distribution.

관련 논문 27편

IntentQA

IntentQA 설명 보기

We contribute an IntentQA dataset with diverse intents in daily social activities. We utilize NExT-QA as the source dataset to construct our dataset. NExT-QA dataset is a comprehensive VideoQA dataset with rich natural daily social activities and detailed QA annotations. Originally, the NExT-QA dataset categorizes itself into three types, i.e., Causal, Temporal, Descriptive. We select the inference QA types, i.e., Causal and Temporal, rather than the factoid Descriptive, to build our IntentQA dataset. Particularly, we select both the Causal Why and Causal How subtypes under Causal, and the Temporal Previous and Temporal Next subtypes under Temporal. The Causal Why (CW) QA usually takes the form of ‘Why [action]? For [intent]’, with the key action appearing in the question and the intent in the answer. On the contrary, the Causal How (CH) QA usually takes the form of ‘How [intent]? By [action]’, with the key action appearing in the answer and the intent in the question. The Temporal Previous (TP) QA usually takes the form of ‘What [action A] before [action B]? ’, while the Temporal Next (TN) QA takes the form of ‘What [action B] after [action A]? ’. In the TP&TN QA, the intent is not explicitly expressed in the question nor answer, but is the implicit causal factor linking the two sequential actions.

관련 논문 27편

IntrA

IntrA 설명 보기

IntrA is an open-access 3D intracranial aneurysm dataset that makes the application of points-based and mesh-based classification and segmentation models available. This dataset can be used to diagnose intracranial aneurysms and to extract the neck for a clipping operation in medicine and other areas of deep learning, such as normal estimation and surface reconstruction. 103 3D models of entire brain vessels are collected by reconstructing scanned 2D MRA images of patients (the raw 2D MRA images are not published due to medical ethics). 1909 blood vessel segments are generated automatically from the complete models, including 1694 healthy vessel segments and 215 aneurysm segments for diagnosis. 116 aneurysm segments are divided and annotated manually by medical experts; the scale of each aneurysm segment is based on the need for a preoperative examination. Geodesic distance matrices are computed and included for each annotated 3D segment, because the expression of the geodesic distance is more accurate than Euclidean distance according to the shape of vessels. Source: https://github.com/intra3d2019/IntrA Image Source: https://github.com/intra3d2019/IntrA

관련 논문 27편

Korean DeepFake Detection Dataset

KoDF
KoDF 설명 보기

The Korean DeepFake Detection Dataset (KoDF) is a large-scale collection of synthesized and real videos focused on Korean subjects, used for the task of deepfake detection. The dataset consists of 62,166 real videos and 175,776 fake videos from 403 subjects. The fake videos are created using 6 different methods: FaceSwap, DeepFaceLab, FSGAN, FOMM, ATFHP and Wav2Lip.

관련 논문 27편

Abstract Meaning Representation (AMR) Annotation Release 2.0

LDC2017T10
LDC2017T10 설명 보기

Abstract Meaning Representation (AMR) Annotation Release 2.0 was developed by the Linguistic Data Consortium (LDC), SDL/Language Weaver, Inc., the University of Colorado's Computational Language and Educational Research group and the Information Sciences Institute at the University of Southern California. It contains a sembank (semantic treebank) of over 39,260 English natural language sentences from broadcast conversations, newswire, weblogs and web discussion forums. AMR captures “who is doing what to whom” in a sentence. Each sentence is paired with a graph that represents its whole-sentence meaning in a tree-structure. AMR utilizes PropBank frames, non-core semantic roles, within-sentence coreference, named entity annotation, modality, negation, questions, quantities, and so on to represent the semantic structure of a sentence largely independent of its syntax.

관련 논문 27편

MedMNIST v2

MedMNIST v2 설명 보기

MedMNIST v2 is a large-scale MNIST-like collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D. All images are pre-processed into 28 x 28 (2D) or 28 x 28 x 28 (3D) with the corresponding classification labels, so that no background knowledge is required for users. Covering primary data modalities in biomedical images, MedMNIST v2 is designed to perform classification on lightweight 2D and 3D images with various data scales (from 100 to 100,000) and diverse tasks (binary/multi-class, ordinal regression and multi-label). The resulting dataset, consisting of 708,069 2D images and 10,214 3D images in total, could support numerous research / educational purposes in biomedical image analysis, computer vision and machine learning. Description and image from: MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification Each subset keeps the same license as that of the source dataset. Please also cite the corresponding paper of source data if you use any subset of MedMNIST.

관련 논문 27편

Multi-Modal CelebA-HQ

Multi-Modal CelebA-HQ 설명 보기

Multi-Modal-CelebA-HQ is a large-scale face image dataset that has 30,000 high-resolution face images selected from the CelebA dataset by following CelebA-HQ. Each image has high-quality segmentation mask, sketch, descriptive text, and image with transparent background. Multi-Modal-CelebA-HQ can be used to train and evaluate algorithms of text-to-image-generation, text-guided image manipulation, sketch-to-image generation, and GANs for face generation and editing. Source: Multi-Modal CelebA-HQ Dataset Image Source: Xia et al

관련 논문 27편

NoW Benchmark

NoW Benchmark 설명 보기

The goal of this benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods under variations in viewing angle, lighting, and common occlusions. The dataset contains 2054 2D images of 100 subjects, captured with an iPhone X, and a separate 3D head scan for each subject. This head scan serves as ground truth for the evaluation. The subjects are selected to contain variations in age, BMI, and sex (55 female, 45 male).

관련 논문 27편

ProofNet

ProofNet 설명 보기

ProofNet is a benchmark for autoformalization and formal proving of undergraduate-level mathematics. The ProofNet benchmarks consists of 371 examples, each consisting of a formal theorem statement in Lean 3, a natural language theorem statement, and natural language proof. The problems are primarily drawn from popular undergraduate pure mathematics textbooks and cover topics such as real and complex analysis, linear algebra, abstract algebra, and topology. Source: ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics Image Source: https://arxiv.org/pdf/2302.12433v1.pdf

관련 논문 27편

Ryerson Audio-Visual Database of Emotional Speech and Song

RAVDESS
RAVDESS 설명 보기

The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) contains 7,356 files (total size: 24.8 GB). The database contains 24 professional actors (12 female, 12 male), vocalizing two lexically-matched statements in a neutral North American accent. Speech includes calm, happy, sad, angry, fearful, surprise, and disgust expressions, and song contains calm, happy, sad, angry, and fearful emotions. Each expression is produced at two levels of emotional intensity (normal, strong), with an additional neutral expression. All conditions are available in three modality formats: Audio-only (16bit, 48kHz .wav), Audio-Video (720p H.264, AAC 48kHz, .mp4), and Video-only (no sound). Note, there are no song files for Actor_18. Paper: The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English Source:

관련 논문 27편

ROAD: The ROad event Awareness Dataset for Autonomous Driving

ROAD
ROAD 설명 보기

ROAD is designed to test an autonomous vehicle's ability to detect road events, defined as triplets composed by an active agent, the action(s) it performs and the corresponding scene locations. ROAD comprises videos originally from the Oxford RobotCar Dataset, annotated with bounding boxes showing the location in the image plane of each road event.

관련 논문 27편

RadarScenes

RadarScenes 설명 보기

RadarScenes is a real-world radar point cloud dataset for automotive applications. It consists of measurements and point-wise annotations from more than four hours of driving collected by four series radar sensors mounted on one test vehicle. Individual detections of dynamic objects were manually grouped to clusters and labeled afterwards. The purpose of this data set is to enable the development of novel (machine learning-based) radar perception algorithms with the focus on moving road users. Images of the recorded sequences were captured using a documentary camera.

관련 논문 27편

S2Looking

S2Looking 설명 보기

S2Looking is a building change detection dataset that contains large-scale side-looking satellite images captured at varying off-nadir angles. The S2Looking dataset consists of 5,000 registered bitemporal image pairs (size of 1024*1024, 0.5 ~ 0.8 m/pixel) of rural areas throughout the world and more than 65,920 annotated change instances. We provide two label maps to separately indicate the newly built and demolished building regions for each sample in the dataset. We establish a benchmark task based on this dataset, i.e., identifying the pixel-level building changes in the bi-temporal images.

관련 논문 27편

CalTech 101 Silhouettes

Silhouettes
Silhouettes 설명 보기

The Caltech 101 Silhouettes dataset consists of 4,100 training samples, 2,264 validation samples and 2,307 test samples. The datast is based on CalTech 101 image annotations. Each image in the CalTech 101 data set includes a high-quality polygon outline of the primary object in the scene. To create the CalTech 101 Silhouettes data set, the authors center and scale each outline and render it on a DxD pixel image-plane. The outline is rendered as a filled, black polygon on a white background. Many object classes exhibit silhouettes that have distinctive class-specific features. A relatively small number of classes like soccer ball, pizza, stop sign, and yin-yang are indistinguishable based on shape, but have been left-in in the data. Source: 1 Introduction Image Source: https://people.cs.umass.edu/~marlin/data.shtml

관련 논문 27편

Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation

UVO
UVO 설명 보기

UVO is a new benchmark for open-world class-agnostic object segmentation in videos. Besides shifting the problem focus to the open-world setup, UVO is significantly larger, providing approximately 8 times more videos compared with DAVIS, and 7 times more mask (instance) annotations per video compared with YouTube-VOS and YouTube-VIS. UVO is also more challenging as it includes many videos with crowded scenes and complex background motions. Some highlights of the dataset include: - High quality instance masks densely annotated at 30 fps on 1024 YouTube videos and 1fps on 10337 videos from Kinetics dataset - Open-world: annotating all objects in each video, 13.5 objects per video on average - Diverse object categories: 57% of objects are not covered by COCO categories

관련 논문 27편

Vid4

Vid4 설명 보기

The Vid4 dataset is generally used for testing video super-resolution. It consists of four sequences: walk (740x480, 47 frames), foliage (740x480, 49 frames), city (704x576, 34 frames), and calendar (720x576, 41 frames).

관련 논문 27편

iNaturalist 2021

iNat2021
iNat2021 설명 보기

iNat2021 is a large-scale image dataset collected and annotated by community scientists that contains over 2.7M images from 10k different species. To make the dataset more accessible the authors have also created a "mini" training dataset with 50 examples per species for a total of 500K images. Each species has 10 validation images, for a total of 100k validation images. There are a total of 500,000 test images. In addition to its overall scale, the main distinguishing feature of iNat2021 is that it contains at least 152 images in the training set for each species.

관련 논문 26편

AdversarialQA

AdversarialQA 설명 보기

We have created three new Reading Comprehension datasets constructed using an adversarial model-in-the-loop. We use three different models; BiDAF (Seo et al., 2016), BERTLarge (Devlin et al., 2018), and RoBERTaLarge (Liu et al., 2019) in the annotation loop and construct three datasets; D(BiDAF), D(BERT), and D(RoBERTa), each with 10,000 training examples, 1,000 validation, and 1,000 test examples. The adversarial human annotation paradigm ensures that these datasets consist of questions that current state-of-the-art models (at least the ones used as adversaries in the annotation loop) find challenging. The three AdversarialQA round 1 datasets provide a training and evaluation resource for such methods.

관련 논문 26편

AlignBench

AlignBench 설명 보기

AlignBench is a comprehensive benchmark designed specifically for evaluating the alignment performance of large Chinese language models (LLMs). It focuses on assessing how well these models align with human intent across multiple dimensions. Let me provide you with more details: 1. Purpose and Importance: - For fine-tuned LLMs, alignment with human intent has become a critical factor in their practical applications. - Existing evaluation benchmarks do not accurately reflect model performance in real-world scenarios or their alignment with human intent. - AlignBench aims to address this challenge by providing a comprehensive, multi-dimensional evaluation benchmark specifically for Chinese LLMs. 2. Data and Construction: - AlignBench uses a human-in-the-loop data curation process to ensure dynamic and realistic evaluation data. - The data comes from real user queries (partly challenging questions constructed by researchers) in ChatGLM online services. - It covers various categories, including fundamental language ability, Chinese understanding, open-ended questions, writing ability, logical reasoning, mathematics, task-oriented role play, and professional knowledge. 3. Evaluation Methodology: - AlignBench employs a multi-dimensional, rule-calibrated evaluation method called "LLM-as-Judge." - It combines human judgment with a chain-of-thought analysis to enhance reliability and interpretability. - The evaluation process includes comparing model responses to high-quality reference answers and generating multi-dimensional scores. 4. CritiqueLLM: - To facilitate easy alignment assessment for Chinese researchers, AlignBench introduces a dedicated evaluation model called CritiqueLLM. - CritiqueLLM can recover 95% of GPT-4's evaluation capability and will provide an accessible API for researchers in the future. (1) THUDM/AlignBench: 多维度中文对齐评测基准 - GitHub. https://github.com/THUDM/AlignBench. (2) 智谱AI发布中文 LLM 对齐评测基准AlignBench | 前途科技. https://accesspath.com/ai/5890084/. (3) AlignBench:量身打造的中文大语言模型对齐评测 - CSDN博客. https://blog.csdn.net/cenyk1230/article/details/135228409. (4) AlignBench:专为「中文 LLM」而生的对齐评测 - 知乎. https://zhuanlan.zhihu.com/p/671884106. (5) AlignBench: Benchmarking Chinese Alignment of Large Language Models. https://arxiv.org/abs/2311.18743.

관련 논문 26편

Animal Kingdom

Animal Kingdom 설명 보기

Animal Kingdom is a large and diverse dataset that provides multiple annotated tasks to enable a more thorough understanding of natural animal behaviors. The wild animal footage used in the dataset records different times of the day in an extensive range of environments containing variations in backgrounds, viewpoints, illumination and weather conditions. More specifically, the dataset contains 50 hours of annotated videos to localize relevant animal behavior segments in long videos for the video grounding task, 30K video sequences for the fine-grained multi-label action recognition task, and 33K frames for the pose estimation task, which correspond to a diverse range of animals with 850 species across 6 major animal classes.

관련 논문 26편

Captured Raw Video Denoising

CRVD
CRVD 설명 보기

The CRVD dataset consists of 55 groups of noisy-clean videos with ISO values ranging from 1600 to 25600. Source: Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes

관련 논문 26편

CVSS

CVSS 설명 보기

CVSS is a massively multilingual-to-English speech to speech translation (S2ST) corpus, covering sentence-level parallel S2ST pairs from 21 languages into English. CVSS is derived from the Common Voice speech corpus and the CoVoST 2 speech-to-text translation (ST) corpus, by synthesizing the translation text from CoVoST 2 into speech using state-of-the-art TTS systems

관련 논문 26편

Cityscapes-VPS

Cityscapes-VPS 설명 보기

Cityscapes-VPS is a video extension of the Cityscapes validation split. It provides 2500-frame panoptic labels that temporally extend the 500 Cityscapes image-panoptic labels. There are total 3000-frame panoptic labels which correspond to 5, 10, 15, 20, 25, and 30th frames of each 500 videos, where all instance ids are associated over time. It not only supports video panoptic segmentation (VPS) task, but also provides super-set annotations for video semantic segmentation (VSS) and video instance segmentation (VIS) tasks.

관련 논문 26편

ContactPose

ContactPose 설명 보기

ContactPose is a dataset of hand-object contact paired with hand pose, object pose, and RGB-D images. ContactPose has 2306 unique grasps of 25 household objects grasped with 2 functional intents by 50 participants, and more than 2.9 M RGB-D grasp images. Source: ContactPose: A Dataset of Grasps with Object Contact and Hand Pose

관련 논문 26편

DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentation

DALES
DALES 설명 보기

We present the Dayton Annotated LiDAR Earth Scan (DALES) data set, a new large-scale aerial LiDAR data set with over a half-billion hand-labeled points spanning 10 square kilometers of area and eight object categories. Large annotated point cloud data sets have become the standard for evaluating deep learning methods. However, most of the existing data sets focus on data collected from a mobile or terrestrial scanner with few focusing on aerial data. Point cloud data collected from an Aerial Laser Scanner (ALS) presents a new set of challenges and applications in areas such as 3D urban modeling and large-scale surveillance. DALES is the most extensive publicly available ALS data set with over 400 times the number of points and six times the resolution of other currently available annotated aerial point cloud data sets. This data set gives a critical number of expert verified hand-labeled points for the evaluation of new 3D deep learning algorithms, helping to expand the focus of current algorithms to aerial data. We describe the nature of our data, annotation workflow, and provide a benchmark of current state-of-the-art algorithm performance on the DALES data set.

관련 논문 26편

DIV2K Random Kernel

DIV2KRK
DIV2KRK 설명 보기

Using the validation set (100 images) from the widely used DIV2K dataset, we blurred and subsampled each image with a different, randomly generated kernel. Kernels were 11x11 anisotropic gaussians with random lengths λ1, λ2∼U(0.6, 5) independently distributed for each axis, rotated by a random angle θ∼U[−π, π].

관련 논문 26편

Document UnderstanDing of Everything

DUDE
DUDE 설명 보기

DUDE is formulated as an instance of Document Question Answering (DocQA) to evaluate how well current solutions deal with multi-page documents, if they can navigate and reason over the layout, and if they can generalize these skills to different document types and domains. Since we cannot provide question-answer pairs about, e.g., ticked checkboxes, on each document instance or document type, the challenge presented by DUDE is characterized equally as a Multi-Domain Long-Tailed Recognition problem Competition website: https://rrc.cvc.uab.es/?ch=23

관련 논문 26편

Drive&Act

Drive&Act 설명 보기

The Drive&Act dataset is a state of the art multi modal benchmark for driver behavior recognition. The dataset includes 3D skeletons in addition to frame-wise hierarchical labels of 9.6 Million frames captured by 6 different views and 3 modalities (RGB, IR and depth). It offers following key features: * 12h of video data in 29 long sequences * Calibrated multi view camera system with 5 views * Multi modal videos: NIR, Depth and Color data * Markerless motion capture: 3D Body Pose and Head Pose * Model of the static interior of the car * 83 manually annotated hierarchical activity labels: * Level 1: Long running tasks (12) * Level 2: Semantic actions (34) * Level 3: Object Interaction tripplets [action|object|location] (6|17|14) Source: Drive&Act: A Multi-Modal Dataset for Fine-Grained Driver Behavior Recognition in Autonomous Vehicles

관련 논문 26편

FairytaleQA

FairytaleQA 설명 보기

FairytaleQA is a dataset focusing on narrative comprehension of kindergarten to eighth-grade students. Annotated by educational experts based on an evidence-based theoretical framework, FairytaleQA consists of 10,580 explicit and implicit questions derived from 278 children-friendly story narratives, covering seven types of narrative elements or relations. It can support narrative Question Generation (QG) and Narrative Question Answering (QA) tasks.

관련 논문 26편

FlickrLogos-32

FlickrLogos-32 설명 보기

Object detection benchmark for logo detection. Images are natural scenes. Each image contains multiple objects, and each image has a total of 1 logo. Logo detection & classification labels are provided.

관련 논문 26편

GenericsKB

GenericsKB 설명 보기

The GenericsKB contains 3.4M+ generic sentences about the world, i.e., sentences expressing general truths such as "Dogs bark," and "Trees remove carbon dioxide from the atmosphere." Generics are potentially useful as a knowledge source for AI systems requiring general world knowledge. The GenericsKB is the first large-scale resource containing naturally occurring generic sentences (as opposed to extracted or crowdsourced triples), and is rich in high-quality, general, semantically complete statements. Generics were primarily extracted from three large text sources, namely the Waterloo Corpus, selected parts of Simple Wikipedia, and the ARC Corpus. A filtered, high-quality subset is also available in GenericsKB-Best, containing 1,020,868 sentences. Source: Allen Institute for AI