Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 87편

Embodied Question Answering

EQA
EQA 설명 보기

The EQA (Embodied Question Answering) dataset is a dataset of visual questions and answers grounded in House3D. For this dataset an agent is spawned at a random location in a 3D environment and asked a question (for e.g. "What color is the car?"). In order to answer, the agent must first intelligently navigate to explore the environment, gather necessary visual information through first-person vision, and then answer the question ("orange").

관련 논문 483편

CSQA

CommonsenseQA
CommonsenseQA 설명 보기

The CommonsenseQA is a dataset for commonsense question answering task. The dataset consists of 12,247 questions with 5 choices each. The dataset was generated by Amazon Mechanical Turk workers in the following process (an example is provided in parentheses): 1. a crowd worker observes a source concept from ConceptNet (“River”) and three target concepts (“Waterfall”, “Bridge”, “Valley”) that are all related by the same ConceptNet relation (“AtLocation”), 2. the worker authors three questions, one per target concept, such that only that particular target concept is the answer, while the other two distractor concepts are not, (“Where on a river can you hold a cup upright to catch water on a sunny day?”, “Where can I stand on a river to see water falling without getting wet?”, “I’m crossing the river, my feet are wet but my body is dry, where am I?”) 3. for each question, another worker chooses one additional distractor from Concept Net (“pebble”, “stream”, “bank”), and the author another distractor (“mountain”, “bottom”, “island”) manually. Source: CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Image Source: CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

관련 논문 339편

Science Question Answering

ScienceQA
ScienceQA 설명 보기

Science Question Answering (ScienceQA) is a new benchmark that consists of 21,208 multimodal multiple choice questions with diverse science topics and annotations of their answers with corresponding lectures and explanations. Out of the questions in ScienceQA, 10,332 (48.7%) have an image context, 10,220 (48.2%) have a text context, and 6,532 (30.8%) have both. Most questions are annotated with grounded lectures (83.9%) and detailed explanations (90.5%). The lecture and explanation provide general external knowledge and specific reasons, respectively, for arriving at the correct answer. To the best of our knowledge, ScienceQA is the first large-scale multimodal dataset that annotates lectures and explanations for the answers. ScienceQA, in contrast to previous datasets, has richer domain diversity from three subjects: natural science, language science, and social science. Questions in each subject are categorized first by the topic (Biology, Physics, Chemistry, etc.), then by the category (Plants, Cells, Animals, etc.), and finally by the skill (Classify fruits and vegetables as plant parts, Identify countries of Africa, etc.). ScienceQA features 26 topics, 127 categories, and 379 skills that cover a wide range of domains.

관련 논문 86편

MovieQA

MovieQA 설명 보기

The MovieQA dataset is a dataset for movie question answering. to evaluate automatic story comprehension from both video and text. The data set consists of almost 15,000 multiple choice question answers obtained from over 400 movies and features high semantic diversity. Each question comes with a set of five highly plausible answers; only one of which is correct. The questions can be answered using multiple sources of information: movie clips, plots, subtitles, and for a subset scripts and DVS. Source: Movie Question Answering: Remembering the Textual Cues for Layered Visual Contents Image Source: https://www.researchgate.net/figure/Examples-of-multiple-choice-QA-from-the-MovieQA-dataset-Each-question-has-5_fig2_321379716

관련 논문 61편

FigureQA

FigureQA 설명 보기

FigureQA is a visual reasoning corpus of over one million question-answer pairs grounded in over 100,000 images. The images are synthetic, scientific-style figures from five classes: line plots, dot-line plots, vertical and horizontal bar graphs, and pie charts. Source: FigureQA: An Annotated Figure Dataset for Visual Reasoning

관련 논문 50편

Explanations for CommonsenseQA

ECQA
ECQA 설명 보기

This repository contains the publicly released dataset, code, and models for the Explanations for CommonsenseQA paper presented at ACL-IJCNLP 2021. Directories data and code inside the root folder contain dataset and code, respectively. The same data and code are also made available through our AIHN collaboration partner institute IIT Delhi. You can download the full paper from here. Note that these annotations are provided for the questions of the CommonsenseQA data (https://www.tau-nlp.org/commonsenseqa): arXiv:1811.00937 [cs.CL] (or arXiv:1811.00937v2 [cs.CL] for this version). ### Citations Please consider citing this paper as follows: @inproceedings{aggarwaletal2021ecqa, title={{E}xplanations for {C}ommonsense{QA}: {N}ew {D}ataset and {M}odels}, author={Shourya Aggarwal and Divyanshu Mandowara and Vishwajeet Agrawal and Dinesh Khandelwal and Parag Singla and Dinesh Garg}, booktitle="Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)}", Pages = 3050–3065, year = "2021", publisher = "Association for Computational Linguistics" }

관련 논문 38편

InsuranceQA

InsuranceQA 설명 보기

InsuranceQA is a question answering dataset for the insurance domain, the data stemming from the website Insurance Library. There are 12,889 questions and 21,325 answers in the training set. There are 2,000 questions and 3,354 answers in the validation set. There are 2,000 questions and 3,308 answers in the test set. Source: APPLYING DEEP LEARNING TO ANSWER SELECTION: A STUDY AND AN OPEN TASK

관련 논문 26편

FairytaleQA

FairytaleQA 설명 보기

FairytaleQA is a dataset focusing on narrative comprehension of kindergarten to eighth-grade students. Annotated by educational experts based on an evidence-based theoretical framework, FairytaleQA consists of 10,580 explicit and implicit questions derived from 278 children-friendly story narratives, covering seven types of narrative elements or relations. It can support narrative Question Generation (QG) and Narrative Question Answering (QA) tasks.

관련 논문 24편

RecipeQA

RecipeQA 설명 보기

RecipeQA is a dataset for multimodal comprehension of cooking recipes. It consists of over 36K question-answer pairs automatically generated from approximately 20K unique recipes with step-by-step instructions and images. Each question in RecipeQA involves multiple modalities such as titles, descriptions or images, and working towards an answer requires (i) joint understanding of images and text, (ii) capturing the temporal flow of events, and (iii) making sense of procedural knowledge. Source: RecipeQA

관련 논문 22편

Time-Sensitive QA

TimeQA
TimeQA 설명 보기

This dataset is aimed to study the existing reading comprehension models' capability to perform temporal reasoning, and see whether they are sensitive to the temporal description in the given question.

관련 논문 17편

FreebaseQA

FreebaseQA 설명 보기

FreebaseQA is a data set for open-domain QA over the Freebase knowledge graph. The question-answer pairs in this data set are collected from various sources, including the TriviaQA data set and other trivia websites (QuizBalls, QuizZone, KnowQuiz), and are matched against Freebase to generate relevant subject-predicate-object triples that were further verified by human annotators. As all questions in FreebaseQA are composed independently for human contestants in various trivia-like competitions, this data set shows richer linguistic variation and complexity than existing QA data sets, making it a good test-bed for emerging KB-QA systems. Source: FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase

관련 논문 13편

eQASC

eQASC 설명 보기

This dataset contains 98k 2-hop explanations for questions in the QASC dataset, with annotations indicating if they are valid (~25k) or invalid (~73k) explanations. This repository addresses the current lack of training data for distinguish valid multihop explanations from invalid, by providing three new datasets. The main one, eQASC, contains 98k explanation annotations for the multihop question answering dataset QASC, and is the first that annotates multiple candidate explanations for each answer. The second dataset, eQASC-perturbed, is constructed by crowd-sourcing perturbations (while preserving their validity) of a subset of explanations in QASC, to test consistency and generalization of explanation prediction models. The third dataset eOBQA is constructed by adding explanation annotations to the OBQA dataset to test generalization of models trained on eQASC. Source: Allen Institute for AI

관련 논문 3편

Long-form Legal Question Answering

LLeQA
LLeQA 설명 보기

LLeQA is a French native dataset for studying information retrieval and long-form question answering in the legal domain. It consists of a knowledge corpus of 27,941 statutory articles collected from the Belgian legislation, and 1,868 legal questions posed by Belgian citizens and labeled by experienced jurists with a comprehensive answer rooted in relevant articles from the corpus.

관련 논문 2편

MultiReQA

MultiReQA 설명 보기

MultiReQA is a cross-domain evaluation for retrieval question answering models. Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus. MultiReQA is a new multi-domain ReQA evaluation suite composed of eight retrieval QA tasks drawn from publicly available QA datasets from the MRQA shared task. MultiReQA contains the sentence boundary annotation from eight publicly available QA datasets including SearchQA, TriviaQA, HotpotQA, NaturalQuestions, SQuAD, BioASQ, RelationExtraction, and TextbookQA. Five of these datasets, including SearchQA, TriviaQA, HotpotQA, NaturalQuestions, SQuAD, contain both training and test data, and three, in cluding BioASQ, RelationExtraction, TextbookQA, contain only the test data. Source: https://github.com/google-research-datasets/MultiReQA

관련 논문 1편

GameQA-140K

GameQA
GameQA 설명 보기

GameQA is a large-scale, diverse, and challenging multimodal reasoning dataset designed to enhance the general reasoning capabilities of Vision Language Models (VLMs). Generated using the innovative Code2Logic framework, it leverages game code to synthesize high-quality visual-language Chain-of-Thought (CoT) data. The dataset addresses the scarcity of multimodal reasoning data, critical for advancing complex multi-step reasoning in VLMs. Each sample includes visual game state, targeted question, original analysis, augmented step-by-step reasoning (refinement) and final answer, derived from the logical structures inherent in game code. Paper: Code2Logic: Game-Code-Driven Data Synthesis for Enhancing VLMs General Reasoning Code: https://github.com/tongjingqi/Code2Logic

관련 논문 0편

OpenEQA

OpenEQA 설명 보기

The OpenEQA dataset is a significant contribution in the field of Embodied Question Answering (EQA). Let me provide you with some details: 1. Definition: - Embodied Question Answering (EQA) involves understanding an environment well enough to answer questions about it in natural language. - EQA agents can achieve this understanding through either episodic memory (as seen in agents using smart glasses) or active exploration of the environment (as in the case of mobile robots). 2. OpenEQA Dataset: - OpenEQA is the first open-vocabulary benchmark dataset for EQA that supports both episodic memory and active exploration use cases. - It contains over 1600 high-quality human-generated questions drawn from more than 180 real-world environments. - The dataset consists of question-answer pairs ($Q, A^*$) and corresponding episode histories ($H$). - You can find the question-answer pairs in the file data/open-eqa-v0.json. - To access the episode histories, follow the instructions provided here. 3. Evaluation Protocol: - OpenEQA also provides an automatic evaluation protocol powered by language model-based evaluation (LLM). - This evaluation protocol correlates well with human judgment. 4. Foundation Models Evaluation: - Researchers evaluated several state-of-the-art foundation models, including GPT-4V, using the OpenEQA dataset. - The findings revealed that these models significantly lag behind human-level performance in EQA tasks. 5. Significance: - OpenEQA serves as a straightforward, measurable, and practically relevant benchmark for current-generation foundation models. - It poses a considerable challenge and inspires research at the intersection of Embodied AI, conversational agents, and world models.