Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 13편

The COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation

The COLOSSEUM
The COLOSSEUM 설명 보기

To realize effective large-scale, real-world robotic applications, we must evaluate how well our robot policies adapt to changes in environmental conditions. Unfortunately, a majority of studies evaluate robot performance in environments closely resembling or even identical to the training setup. We present Colosseum, a novel simulation benchmark,with 20 diverse manipulation tasks, that enables systematical evaluation of models across 12 axes of environmental perturbations. These perturbations include changes in color, texture, and size of objects, table-tops, and backgrounds; we also vary lighting, distractors, and camera pose. Using Colosseum, we compare 4 state-of-the-art manipulation models to reveal that their success rate degrades between 30-50% across these perturbation factors. When multiple perturbations are applied in unison, the success rate degrades > 75%. We identify that changing the number of distractor objects, target object color, or lighting conditions are the perturbations that reduce model performance the most. To verify the ecological validity of our results, we show that our results in simulation are correlated (R2 = 0.614) to similar perturbations in real-world experiments. We open source code for others to use Colosseum, and also release code to 3D print the objects used to replicate the real-world perturbations. Ultimately, we hope that Colosseum will serve as a benchmark to identify modeling decisions that systematically improve generalization for manipulation.

관련 논문 13편

Touch and Go

Touch and Go 설명 보기

This dataset encompasses a diverse range of tactile features that are instrumental in bifurcating various material properties. Three downstream tasks are considered: 1) categorization of materials, 2) distinction between hard and soft surfaces, and 3) distinction between smooth and textured surfaces.

관련 논문 13편

User Preference-aware Fake News Detection

UPFD
UPFD 설명 보기

For benchmarking, please refer to its variant UPFD-POL and UPFD-GOS. The dataset has been integrated with Pytorch Geometric (PyG) and Deep Graph Library (DGL). You can load the dataset after installing the latest versions of PyG or DGL. The UPFD dataset includes two sets of tree-structured graphs curated for evaluating binary graph classification, graph anomaly detection, and fake/real news detection tasks. The dataset is dumped in the form of Pytorch-Geometric dataset object. You can easily load the data and run various GNN models using PyG. The dataset includes fake&real news propagation (retweet) networks on Twitter built according to fact-check information from Politifact and Gossipcop. The news retweet graphs were originally extracted by FakeNewsNet. Each graph is a hierarchical tree-structured graph where the root node represents the news; the leaf nodes are Twitter users who retweeted the root news. A user node has an edge to the news node if he/she retweeted the news tweet. Two user nodes have an edge if one user retweeted the news tweet from the other user. We crawled near 20 million historical tweets from users who participated in fake news propagation in FakeNewsNet to generate node features in the dataset. We incorporate four node feature types in the dataset, the 768-dimensional bert and 300-dimensional spacy features are encoded using pretrained BERT and spaCy word2vec, respectively. The 10-dimensional profile feature is obtained from a Twitter account's profile. You can refer to profile_feature.py for profile feature extraction. The 310-dimensional content feature is composed of a 300-dimensional user comment word2vec (spaCy) embedding plus a 10-dimensional profile feature. The dataset statistics is shown below: | Data | #Graphs | #Fake News| #Total Nodes | #Total Edges | #Avg. Nodes per Graph | |-------|--------|--------|--------|--------|--------| | Politifact | 314 | 157 | 41,054 | 40,740 | 131 | | Gossipcop | 5464 | 2732 | 314,262 | 308,798 | 58 | Please refer to the paper for more details about the UPFD dataset. Due to the Twitter policy, we could not release the crawled user's historical tweets publicly. To get the corresponding Twitter user information, you can refer to the news lists under \data in our github repo and map the news id to FakeNewsNet. Then, you can crawl the user information by following the instruction on FakeNewsNet. In the UPFD project, we use Tweepy and Twitter Developer API to get the user information.

관련 논문 13편

UnAV-100

UnAV-100 설명 보기

Existing audio-visual event localization (AVE) handles manually trimmed videos with only a single instance in each of them. However, this setting is unrealistic as natural videos often contain numerous audio-visual events with different categories. To better adapt to real-life applications, we focus on the task of dense-localizing audio-visual events, which aims to jointly localize and recognize all audio-visual events occurring in an untrimmed video. To tackle this problem, we introduce the first Untrimmed Audio-Visual (UnAV-100) dataset, which contains 10K untrimmed videos with over 30K audio-visual events covering 100 event categories. Each video has 2.8 audio-visual events on average, and the events are usually related to each other and might co-occur as in real-life scenes. We believe our UnAV-100, with its realistic complexity, can promote the exploration on comprehensive audio-visual video understanding.

관련 논문 13편

VQA-CP

VQA-CP 설명 보기

The VQA-CP dataset was constructed by reorganizing VQA v2 such that the correlation between the question type and correct answer differs in the training and test splits. For example, the most common answer to questions starting with What sport… is tennis in the training set, but skiing in the test set. A model that guesses an answer primarily from the question will perform poorly. Source: Unshuffling Data for Improved Generalization Image Source: https://arxiv.org/pdf/1712.00377.pdf

관련 논문 13편

ViQuAE

ViQuAE 설명 보기

ViQuAE is a dataset for KVQAE (Knowledge-based Visual Question Answering about named Entities), a task which consists in answering questions about named entities grounded in a visual context using a Knowledge Base. It is the first KVQAE dataset to cover a wide range of entity types (e.g. persons, landmarks, and products). We argue that KVQAE is a clear, well-defined task that can be evaluated easily, making it suitable to track the progress of multimodal entity representation’s quality. Multimodal entity representation is a central issue that will allow to make human-machine interactions more natural. For example, while watching a movie, one might wonder ‘‘Where did I already see this actress?’’ or ‘‘Did she ever win an Oscar?’’

관련 논문 13편

Visual Document Retrieval Benchmark

Vidore
Vidore 설명 보기

It is collection regrouping all datasets constituting the ViDoRe benchmark. It includes the test sets from different academic datasets (ArXiVQA, DocVQA, InfoVQA, TATDQA, TabFQuAD) and from datasets synthetically generated spanning various themes and industrial applications: (Artificial Intelligence, Government Reports, Healthcare Industry, Energy and Shift Project). Further details can be found on the corresponding dataset cards.

관련 논문 13편

Visual Madlibs

Visual Madlibs 설명 보기

Visual Madlibs is a dataset consisting of 360,001 focused natural language descriptions for 10,738 images. This dataset is collected using automatically produced fill-in-the-blank templates designed to gather targeted descriptions about: people and objects, their appearances, activities, and interactions, as well as inferences about the general scene or its broader context. Source: Visual Madlibs: Fill in the blank Image Generation and Question Answering Image Source: Yu et al

관련 논문 13편

Watch-n-Patch

Watch-n-Patch 설명 보기

The Watch-n-Patch dataset was created with the focus on modeling human activities, comprising multiple actions in a completely unsupervised setting. It is collected with Microsoft Kinect One sensor for a total length of about 230 minutes, divided in 458 videos. 7 subjects perform human daily activities in 8 offices and 5 kitchens with complex backgrounds. Moreover, skeleton data are provided as ground truth annotations. Source: Head Detection with Depth Images in the Wild Image Source: https://openaccess.thecvf.com/content_cvpr_2015/papers/Wu_Watch-n-Patch_Unsupervised_Understanding_2015_CVPR_paper.pdf

관련 논문 13편

WaterScenes

WaterScenes 설명 보기

A Multi-Task 4D Radar-Camera Fusion Dataset for Autonomous Driving on Water Surfaces description of the dataset * WaterScenes, the first multi-task 4D radar-camera fusion dataset on water surfaces, which offers data from multiple sensors, including a 4D radar, monocular camera, GPS, and IMU. It can be applied in multiple tasks, such as object detection, instance segmentation, semantic segmentation, free-space segmentation, and waterline segmentation. * Our dataset covers diverse time conditions (daytime, nightfall, night), lighting conditions (normal, dim, strong), weather conditions (sunny, overcast, rainy, snowy) and waterway conditions (river, lake, canal, moat). An information list is also offered for retrieving specific data for experiments under different conditions. * We provide 2D box-level and pixel-level annotations for camera images, and 3D point-level annotations for radar point clouds. We also offer precise timestamps for the synchronization of different sensors, as well as intrinsic and extrinsic parameters. * We provide a toolkit for radar point clouds that includes: pre-processing, labeling, projection and visualization, assisting researchers in processing and analyzing our dataset.

관련 논문 13편

WeatherBench

WeatherBench 설명 보기

A benchmark dataset for data-driven medium-range weather forecasting, a topic of high scientific interest for atmospheric and computer scientists alike. Source: WeatherBench: A benchmark dataset for data-driven weather forecasting

관련 논문 13편

Who did What

Who-did-What
Who-did-What 설명 보기

Who-did-What collects its corpus from news and provides options for questions similar to CBT. Each question is formed from two independent articles: an article is treated as context to be read and a separate article on the same event is used to form the query. Source: ChID: A Large-scale Chinese IDiom Dataset for Cloze Test Image Source: https://tticnlp.github.io/who_did_what/sample.html

관련 논문 13편

YUP++ Dynamic Scenes dataset

YUP++
YUP++ 설명 보기

A new and challenging video database of dynamic scenes that more than doubles the size of those previously available. This dataset is explicitly split into two subsets of equal size that contain videos with and without camera motion to allow for systematic study of how this variable interacts with the defining dynamics of the scene per se. Source: Temporal Residual Networks for Dynamic Scene Recognition

관련 논문 13편

Multi-relational Graph Dataset for Yelp Spam Review Detection

Yelp-Fraud
Yelp-Fraud 설명 보기

Yelp-Fraud is a multi-relational graph dataset built upon the Yelp spam review dataset, which can be used in evaluating graph-based node classification, fraud detection, and anomaly detection models. - Dataset Statistics | # Nodes | %Fraud Nodes (Class=1) | |-------|--------| | 45,954 | 14.5 | | Relation | # Edges | |--------|--------| | R-U-R | 49,315 | | R-T-R | 573,616 | | R-S-R | 3,402,743 | | All | 3,846,979 | - Graph Construction The Yelp spam review dataset includes hotel and restaurant reviews filtered (spam) and recommended (legitimate) by Yelp. We conduct a spam review detection task on the Yelp-Fraud dataset which is a binary classification task. We take 32 handcrafted features from SpEagle paper as the raw node features for Yelp-Fraud. Based on previous studies which show that opinion fraudsters have connections in user, product, review text, and time, we take reviews as nodes in the graph and design three relations: 1) R-U-R: it connects reviews posted by the same user; 2) R-S-R: it connects reviews under the same product with the same star rating (1-5 stars); 3) R-T-R: it connects two reviews under the same product posted in the same month. To download the dataset, please visit this Github repo. For any other questions, please email ytongdou(AT)gmail.com for inquiry.

관련 논문 13편

arXiv Summarization Dataset

arXiv Summarization Dataset 설명 보기

This is a dataset for evaluating summarisation methods for research papers. Source: A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents

관련 논문 13편

eQASC

eQASC 설명 보기

This dataset contains 98k 2-hop explanations for questions in the QASC dataset, with annotations indicating if they are valid (~25k) or invalid (~73k) explanations. This repository addresses the current lack of training data for distinguish valid multihop explanations from invalid, by providing three new datasets. The main one, eQASC, contains 98k explanation annotations for the multihop question answering dataset QASC, and is the first that annotates multiple candidate explanations for each answer. The second dataset, eQASC-perturbed, is constructed by crowd-sourcing perturbations (while preserving their validity) of a subset of explanations in QASC, to test consistency and generalization of explanation prediction models. The third dataset eOBQA is constructed by adding explanation annotations to the OBQA dataset to test generalization of models trained on eQASC. Source: Allen Institute for AI

관련 논문 13편

ecoset

ecoset 설명 보기

Ecoset, an ecologically motivated image dataset, is a large-scale image dataset designed for human visual neuroscience, which consists of over 1.5 million images from 565 basic-level categories. Category selection was based on English nouns that most frequently occur in spoken language (estimated on a set of 51 million words obtained from American television and film subtitles) and concreteness ratings from human observers. Ecoset consists of basic-level categories (including human categories man, woman, and child) that describe physical things in the world (rather than abstract concepts) that are important to humans.

관련 논문 12편

ACM (Heterogeneous Node Classification)

ACM (Heterogeneous Node Classification) 설명 보기

A popular dataset for node classification on heterogeneous graphs.

관련 논문 12편

ACNE04

ACNE04 설명 보기

The ACNE04 dataset includes 3756 Chinese face images with Acne. The ACNE04 dataset includes the annotations of local lesion numbers and global acne severity based on Hayashi Criterion.

관련 논문 12편

Adam: automatic detection challenge on age-related macular degeneration

ADAM
ADAM 설명 보기

ADAM is organized as a half day Challenge, a Satellite Event of the ISBI 2020 conference in Iowa City, Iowa, USA. The ADAM challenge focuses on the investigation and development of algorithms associated with the diagnosis of Age-related Macular degeneration (AMD) and segmentation of lesions in fundus photos from AMD patients. The goal of the challenge is to evaluate and compare automated algorithms for the detection of AMD on a common dataset of retinal fundus images. We invite the medical image analysis community to participate by developing and testing existing and novel automated fundus classification and segmentation methods. Instructions: ADAM: Automatic Detection challenge on Age-related Macular degeneration Link: https://amd.grand-challenge.org Age-related macular degeneration, abbreviated as AMD, is a degenerative disorder in the macular region. It mainly occurs in people older than 45 years old and its incidence rate is even higher than diabetic retinopathy in the elderly. The etiology of AMD is not fully understood, which could be related to multiple factors, including genetics, chronic photodestruction effect, and nutritional disorder. AMD is classified into Dry AMD and Wet AMD. Dry AMD (also called nonexudative AMD) is not neovascular. It is characterized by progressive atrophy of retinal pigment epithelium (RPE). In the late stage, drusen and the large area of atrophy could be observed under ophthalmoscopy. Wet AMD (also called neovascular or exudative AMD), is characterized by active neovascularization under RPE, subsequently causing exudation, hemorrhage, and scarring, and will eventually cause irreversible damage to the photoreceptors and rapid vision loss if left untreated. An early diagnosis of AMD is crucial to treatment and prognosis. Fundus photo is one of the basic examinations. The current dataset is composed of AMD and non-AMD (myopia, normal control, etc.) photos. Typical signs of AMD that can be found in these photos include drusen, exudation, hemorrhage, etc. The ADAM challenge has 4 tasks: Task 1: Classification of AMD and non-AMD fundus images. Task 2: Detection and segmentation of optic disc. Task 3: Localization of fovea. Task 4: Detection and Segmentation of lesions from fundus images.

관련 논문 12편

AliExpress - 110k

AE-110k
AE-110k 설명 보기

The dataset contains product information from AliExpress Sports & Entertainment category. Each attribute value in "Item Specific" is matched against the product title using exact string match to generate positive triples <title, attribute, value>. Negative triples <title, attribute, NULL> are randomly generated. Each triple is stored in a line and separated by \u0001.

관련 논문 12편

Aesthetic Visual Analysis

Aesthetic Visual Analysis 설명 보기

Aesthetic Visual Analysis is a dataset for aesthetic image assessment that contains over 250,000 images along with a rich variety of meta-data including a large number of aesthetic scores for each image, semantic labels for over 60 categories as well as labels related to photographic style.

관련 논문 12편

AndroZoo

AndroZoo 설명 보기

AndroZoo is a growing collection of Android apps collected from several sources, including the official Google Play app market and a growing collection of various metadata of those collected apps aiming at facilitating the Android-relevant research works. It currently contains 15,097,876 different APKs, each of which has been (or will be) analysed by tens of different AntiVirus products to know which applications are detected as Malware. Each app has over 20 types of metadata such as VirusTotal reports.

관련 논문 12편

Artaxor

Artaxor 설명 보기

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

관련 논문 12편

BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models

BenchLMM
BenchLMM 설명 보기

Large Multimodal Models (LMMs) such as GPT-4V and LLaVA have shown remarkable capabilities in visual reasoning with common image styles. However, their robustness against diverse style shifts, crucial for practical applications, remains largely unexplored. In this paper, we propose a new benchmark, BenchLMM, to assess the robustness of LMMs against three different styles: artistic image style, imaging sensor style, and application style, where each style has five sub-styles. Utilizing BenchLMM, we comprehensively evaluate state-of-the-art LMMs and reveal: 1) LMMs generally suffer performance degradation when working with other styles; 2) An LMM performs better than another model in common style does not guarantee its superior performance in other styles; 3) LMMs' reasoning capability can be enhanced by prompting LMMs to predict the style first, based on which we propose a versatile and training-free method for improving LMMs; 4) An intelligent LMM is expected to interpret the causes of its errors when facing stylistic variations. We hope that our benchmark and analysis can shed new light on developing more intelligent and versatile LMMs.

관련 논문 12편

Bongard-HOI

Bongard-HOI 설명 보기

Bongard-HOI testifies to which extent your few-shot visual learner can quickly induce the true HOI concept from a handful of images and perform reasoning with it. Further, the learner is also expected to transfer the learned few-shot skills to novel HOI concepts compositionally.

관련 논문 12편

Broad Twitter Corpus

Broad Twitter Corpus 설명 보기

This paper introduces the Broad Twitter Corpus (BTC), which is not only significantly bigger, but sampled across different regions, temporal periods, and types of Twitter users. The gold-standard named entity annotations are made by a combination of NLP experts and crowd workers, which enables us to harness crowd recall while maintaining high quality. We also measure the entity drift observed in our dataset (i.e. how entity representation varies over time), and compare to newswire.

관련 논문 12편

Cornell eRulemaking Corpus

CDCP
CDCP 설명 보기

The Cornell eRulemaking Corpus – CDCP is an argument mining corpus annotated with argumentative structure information capturing the evaluability of arguments. The corpus consists of 731 user comments on Consumer Debt Collection Practices (CDCP) rule by the Consumer Financial Protection Bureau (CFPB); the resulting dataset contains 4931 elementary unit and 1221 support relation annotations. It is a resource for building argument mining systems that can not only extract arguments from unstructured text, but also identify what additional information is necessary for readers to understand and evaluate a given argument. Immediate applications include providing real-time feedback to commenters, specifying which types of support for which propositions can be added to construct better-formed arguments.

관련 논문 12편

BioCreative V CDR Task Corpus

CDR
CDR 설명 보기

The BioCreative V CDR task corpus is manually annotated for chemicals, diseases and chemical-induced disease (CID) relations. It contains the titles and abstracts of 1500 PubMed articles and is split into equally sized train, validation and test sets. It is common to first tune a model on the validation set and then train on the combination of the train and validation sets before evaluating on the test set. It is also common to filter negative relations with disease entities that are hypernyms of a corresponding true relations disease entity within the same abstract (see Appendix C of this paper for details).

관련 논문 12편

Contextualized Commonsense Inference in Dialogues

CICERO
CICERO 설명 보기

CICERO contains 53,000 inferences for five commonsense dimensions -- cause, subsequent event, prerequisite, motivation, and emotional reaction -- collected from 5600 dialogues. It involves two challenging generative and multi-choice alternative selection tasks for the state-of-the-art NLP models to solve. Download the dataset using this link.

관련 논문 12편

COCO-MLT

COCO-MLT 설명 보기

The COCO-MLT is created from MS COCO-2017, containing 1,909 images from 80 classes. The maximum of training number per class is 1,128 and the minimum is 6. We use the test set of COCO2017 with 5,000 for evaluation. The ratio of head, medium, and tail classes is 22:33:25 in COCO-MLT.

관련 논문 12편

machine bias risk assessments in criminal sentencing

COMPAS
COMPAS 설명 보기

Dataset used by ProPublica to assess and analyse the fairness of the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) software. compas.db - a sqlite3 database containing criminal history, jail and prison time, demographics and COMPAS risk scores for defendants from Broward County.

관련 논문 12편

COVID19 Fake News Detection in English

COVID-19 Fake News Dataset
COVID-19 Fake News Dataset 설명 보기

Along with COVID-19 pandemic we are also fighting an `infodemic'. Fake news and rumors are rampant on social media. Believing in rumors can cause significant harm. This is further exacerbated at the time of a pandemic. To tackle this, we curate and release a manually annotated dataset of 10,700 social media posts and articles of real and fake news on COVID-19. We benchmark the annotated dataset with four machine learning baselines - Decision Tree, Logistic Regression , Gradient Boost , and Support Vector Machine (SVM). We obtain the best performance of 93.46% F1-score with SVM.

관련 논문 12편

CityStreet

CityStreet 설명 보기

Datasets for multi-view crowd counting in wide-area scenes. Includes our CityStreet dataset, as well as the counting and metadata for multi-view counting on PETS2009 and DukeMTMC. CityStreet is a real-world city scene dataset collected around the intersection of a crowded street. The scene size of the dataset is around 58m×72m. The ground plane map resolution is 320×384.

관련 논문 12편

CosmoFlow

CosmoFlow 설명 보기

The latest CosmoFlow dataset includes around 10,000 cosmological N-body dark matter simulations. The simulations are run using MUSIC to generate the initial conditions, and are evolved with pyCOLA, a multithreaded Python/Cython N-body code. The output of these simulations is then binned into a 3D histogram of particle counts in a cube of size 512x512x512, which is sampled at 4 different redshifts.

관련 논문 12편

DBLP (Heterogeneous Node Classification)

DBLP (Heterogeneous Node Classification) 설명 보기

A popular dataset for node classification on heterogeneous graphs.

관련 논문 12편

Dialog System Technology Challenges Task 1

DSTC7 Task 1
DSTC7 Task 1 설명 보기

The DSTC7 Task 1 dataset is a dataset and task for goal-oriented dialogue. The data originates from human-human conversations, which is built from online resources, specifically the Ubuntu Internet Relay Chat (IRC) channel and an Advising dataset from the University of Michigan. Source: Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems Image Source: https://www.aclweb.org/anthology/W19-4107.pdf

관련 논문 12편

Diverse Weather Dataset

DWD
DWD 설명 보기

Urban-scene detection dataset that consists of five different weather conditions: daytime-sunny, night-sunny, dusk-rainy, daytime-foggy, and night-rainy. The images are collected from diverse weather datasets: Cityscapes, BDD-100k, FoggyCityscapes, and Adverse-Weather. The dataset is used to evaulate the model performance on the single-domain generalized object detection (Single-DGOD). - Source paper: Single-Domain Generalized Object Detection in Urban Scene via Cyclic-Disentangled Self-Distillation - Source page: AmingWu/Single-DGOD - Source dataset: DWD (Diverse Weather Dataset)

관련 논문 12편

DramaQA

DramaQA 설명 보기

The DramaQA focuses on two perspectives: 1) Hierarchical QAs as an evaluation metric based on the cognitive developmental stages of human intelligence. 2) Character-centered video annotations to model local coherence of the story. The dataset is built upon the TV drama "Another Miss Oh" and it contains 17,983 QA pairs from 23,928 various length video clips, with each QA pair belonging to one of four difficulty levels. Source: DramaQA: Character-Centered Video Story Understanding with Hierarchical QA