Papers with Code 아카이브

Question Answering

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
19
평가표
153
보존된 평가 기록
1,781

Question answering can be segmented into domain-specific tasks like community question answering and knowledge-base question answering. Popular benchmark datasets for evaluation question answering systems include SQuAD, HotPotQA, bAbI, TriviaQA, WikiQA, and many others. Models for question answering are typically evaluated on metrics like EM and F1. Some recent top performing models are T5 and XLNet.

( Image credit: SQuAD )

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

ToM QA

The data consists of a set of 3 task types and 4 question types, creating 12 total scenarios. The tasks are grouped into stories, which are denoted by the numbering at the start of each line. Source: ToM QA

TQBA++

Ultra-lightweight, multilingual QA eval dataset for rapid testing LLMs. Dataset Characteristics: - Multilingual: Includes packs for Arabic, Chinese, French, German, Japanese, Korean, Portuguese, Russian, Spanish, and Turkish. - Compact: Contains a curated English gold-standard set of 52 QA pairs (under 20kB), enabling immediate and resource-friendly evaluations. - Synthetic Generation: Features a LiteLLM-powered synthetic data generator see tinyqabenchmarkpp, allowing quick creation of custom evaluation sets tailored to specific domains or languages. - Metadata Support: Provided in Croissant-compatible formats, ready for seamless integration with modern evaluation harnesses and CI tools. Motivation and Content Summary: The primary motivation behind TQB++ is to enable rapid iteration and continuous integration (CI) of language models. Existing evaluation benchmarks typically involve significant computational overhead and slow feedback loops. In contrast, TQB++ offers near-instantaneous assessments of model performance and prompt stability across multiple languages. It is particularly sensitive to issues such as prompt-template regressions, tokenizer drift, and fine-tuning side effects. Potential Use Cases: - Continuous Integration (CI): Immediate detection of breaking changes or regressions in LLM pipelines. - Multilingual Model Validation: Quickly assess model accuracy and performance across multiple languages without large compute costs. - Prompt Optimization and Testing: Ideal for iterative prompt refinement workflows, enabling fast feedback loops and effective tuning. - Teaching and Prototyping: Educational use in courses or workshops, showcasing multilingual LLM evaluation in real-time scenarios.

TUMTraffic-VideoQA

TUMTraffic-VideoQA is a novel dataset designed to understand spatiotemporal video in complex roadside traffic scenarios. The dataset comprises 1,000 videos, featuring 85,000 multiple-choice QA pairs, 2,300 object captioning, and 5,700 object grounding annotations, encompassing diverse real-world conditions such as adverse weather and traffic anomalies. By incorporating tuple-based spatiotemporal object expressions, TUMTraffic-VideoQA unifies three essential tasks—multiple-choice video question answering, referred object captioning, and spatiotemporal object grounding—within a cohesive evaluation framework.

TupleInf Open IE Dataset

The TupleInf Open IE dataset contains Open IE tuples extracted from 263K sentences that were used by the solver in “Answering Complex Questions Using Open Information Extraction” (referred as Tuple KB, T). These sentences were collected from a large Web corpus using training questions from 4th and 8th grade as queries. This dataset contains 156K sentences collected for 4th grade questions and 107K sentences for 8th grade questions. Each sentence is followed by the Open IE v4 tuples using their simple format. Source: Allen Institute for AI

UIT-ViCoQA

UIT-ViCoQA is a new corpus for conversational machine reading comprehension in the Vietnamese language. This corpus consists of 10,000 questions with answers over 2,000 conversations about health news articles.

VCG+112K

Video-ChatGPT introduces the VideoInstruct100K dataset, which employs a semi-automatic annotation pipeline to generate 75K instruction-tuning QA pairs. To address the limitations of this annotation process, we present \ourdata~dataset developed through an improved annotation pipeline. Our approach improves the accuracy and quality of instruction tuning pairs by improving keyframe extraction, leveraging SoTA large multimodal models (LMMs) for detailed descriptions, and refining the instruction generation strategy.

VDQG

The Visual Discriminative Question Generation (VDQG) dataset contains 11202 ambiguous image pairs collected from Visual Genome. Each image pair is annotated with 4.6 discriminative questions and 5.9 non-discriminative questions on average.

Visual Choice of Plausible Alternatives

Visual Choice of Plausible Alternatives (VCOPA) is an evaluation dataset containing 380 VCOPA questions and over 1K images with various topics, which is amenable to automatic evaluation, and present the performance of baseline reasoning approaches as initial benchmarks for future systems. Source: Visual Choice of Plausible Alternatives: An Evaluation of Image-based Commonsense Causal Reasoning

Visual Haystacks (VHs)

Visual Haystacks (VHs) is a "visual-centric" Needle-In-A-Haystack (NIAH) benchmark specifically designed to evaluate the capabilities of Large Multimodal Models (LMMs) in visual retrieval and reasoning over sets of unrelated images. Unlike conventional NIAH challenges that center on text-related retrieval and understanding with limited anecdotal examples, VHs contains a much larger number of examples and focuses on "simple visual tasks", providing a more accurate reflection of LMMs' capabilities when dealing with extensive visual context.

VlogQA

The VlogQA consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube - an extensive source of user-uploaded content, covering the topics of food and travel in the Vietnamese language. This dataset is used for research in Vietnamese Spoken-Based Machine Reading Comprehension.

vqa-nle-llava

VQA NLE synthetic dataset, made with LLaVA-1.5 using features from GQA dataset. Total number of unique datas: 66684 ## Languages eng ## Supported Tasks Question Answering ## Dataset Usage from datasets import load_dataset dset = datasets.load_dataset("patrickamadeus/vqa-nle-llava", name='<CONFIG_NAME>', trust_remote_code=True) ## Dataset Version Source: 1.0.1. Date: 2024.09.25. ## Dataset License CC-BY 4.0 ## Citation If you are using the VQA NLE LLaVA dataloader in your work, please cite the following: @misc{irawan2024efficientrobustvqanledata, title={Towards Efficient and Robust VQA-NLE Data Generation with Large Vision-Language Models}, author={Patrick Amadeus Irawan and Genta Indra Winata and Samuel Cahyawijaya and Ayu Purwarianti}, year={2024}, eprint={2409.14785}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2409.14785}, }

VTQA

VTQA is a dataset containing open-ended questions about image-text pairs. This dataset requires the model to align multimedia representations of the same entity to implement multi-hop reasoning between image and text and finally use natural language to answer the question. The aim of this dataset is to develop and benchmark models that are capable of multimedia entity alignment, multi-step reasoning and open-ended answer generation. VTQA dataset consists of 10,238 image-text pairs and 27,317 questions. The images are real images from MSCOCO dataset, containing a variety of entities. The annotators are required to first annotate relevant text according to the image, and then ask questions based on the image-text pair, and finally answer the question open-ended.

WangchanX-Legal-ThaiCCL-RAG

The WangchanX-Legal-ThaiCCL-RAG dataset supports the development Retrieval-Augmented Generation (RAG) for Thai Legal question answering. This dataset is allows developers to finetune both retrieval model - to better retrieve relevant law section, and Large Language Model (LLM) - for instruction tuning. Our dataset supports Corporate and Commercial Law (thus ThaiCCL name)

WikiQAar

A publicly available set of question and sentence pairs, collected and annotated for research on open-domain question answering. Source: WikiQA: A Challenge Dataset for Open-Domain Question Answering

WikiSuggest

To collect WikiSuggest, Google Suggest API is used to harvest natural language questions and submit them to Google Search. Whenever Google Search returns a box with a short answer from Wikipedia, an example from the question, answer, and the Wikipedia document are created. If the answer string is missing from the document this often implies a spurious question-answer pair, such as (‘what time is half time in rugby’, ‘80 minutes, 40 minutes’). Question-answer pairs without the exact answer string are pruned. Fifty examples after filtering are examined and 54% were found to be well-formed question-answer pairs where answers in the document can be grounded, 20% contained answers without textual evidence in the document (the answer string exists in an irreleveant context), and 26% contain incorrect QA pairs. Source: Coarse-to-Fine Question Answering for Long Documents

WorldCuisines

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data. The dataset can be found in https://huggingface.co/datasets/worldcuisines/vqa

Xamarin Q&A

Xamarin Q&A consists of two datasets of questions and answers for studying the development of cross-platform mobile applications using the Xamarin framework. The two datasets were created by mining two Q&A sites: Xamarin Forum and Stack Overflow. The datasets have 85,908 questions mined from the Xamarin Forum and 44,434 from Stack Overflow.

XLingEval

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

Customer Support on Twitter

The Customer Support on Twitter dataset is a large, modern corpus of tweets and replies to aid innovation in natural language understanding and conversational models, and for study of modern customer support practices and impact.

LMCQA

This dataset contains a set of multiple-choice questions related to various legal topics. The dataset contains 20 questions covering various aspects of legal knowledge, such as the workings of the European Commission, types of legal documents, procedures in the court system, legal definitions, and European Union+United Kingdom law, among others.

Visiting Card | ID Card Images | Hindi-English

This dataset is an extremely challenging set of over 2000+ original Visiting card/ID card images captured and crowdsourced from over 300+ urban and rural areas, where each image is manually reviewed and verified by computer vision professionals at Datacluster Labs. ### Dataset Features - Dataset size : 2000+ - Captured by : Over 150+ crowdsource contributors - Resolution : 100% images HD and above (1920x1080 and above) - Location : Captured with 300+ cities accross India - Diversity : Covers a wide variety of things such as reflective paper, different fonts, different color and type of cards, etc. - Device used : Captured using mobile phones in 2020-2021 - Usage : Visiting card detection, card edge detection, paper edge detection, ID card OCR, Hindi OCR, etc. ### Available Annotation formats COCO, YOLO, PASCAL-VOC, Tf-Record To download full datasets or to submit a request for your dataset needs, please ping us at sales@datacluster.ai Visit www.datacluster.ai to know more. Note: All the images are manually captured and verified by a large contributor base on DataCluster platform

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.