Papers with Code 아카이브

Retrieval

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
3
평가표
11
보존된 평가 기록
19

A methodology that involves selecting relevant data or examples from a large dataset to support tasks like prediction, learning, or inference. It enhances models by providing context or additional information, often used in systems like retrieval-augmented generation or in-context learning.

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

DiSCQ

DiSCQ is a newly curated question dataset composed of 2,000+ questions paired with the snippets of text (triggers) that prompted each question. The questions are generated by medical experts from 100+ MIMIC-III discharge summaries. This dataset is released to facilitate further research into realistic clinical Question Answering (QA) and Question Generation (QG). Source: Learning to Ask Like a Physician Image Source: https://arxiv.org/pdf/2206.02696.pdf

FanOutQA

FanOutQA is a high quality, multi-hop, multi-document benchmark for large language models using English Wikipedia as its knowledge base. Compared to other question-answering benchmarks, FanOutQA requires reasoning over a greater number of documents, with the benchmark's main focus being on the titular fan-out style of question. We present these questions in three tasks -- closed-book, open-book, and evidence-provided -- which measure different abilities of LLM systems.

LLeQA

LLeQA is a French native dataset for studying information retrieval and long-form question answering in the legal domain. It consists of a knowledge corpus of 27,941 statutory articles collected from the Belgian legislation, and 1,868 legal questions posed by Belgian citizens and labeled by experienced jurists with a comprehensive answer rooted in relevant articles from the corpus.

ToolLens

The ToolLens dataset consists of 18,770 concise yet intentionally multifaceted queries, each associated with 1 to 3 verified tools out of a total of 464, designed to better mimic real-world user interactions.

DTGB

We introduce Dynamic Text-attributed Graph Benchmark (DTGB), a collection of large-scale, time-evolving graphs from diverse domains, with nodes and edges enriched by dynamically changing text attributes and categories. To facilitate the use of DTGB, we design standardized evaluation procedures based on four real-world use cases: future link prediction, destination node retrieval, edge classification, and textual relation generation. These tasks require models to understand both dynamic graph structures and natural language, highlighting the unique challenges posed by DyTAGs.

LatamXIX

A novel dataset of 19th-century Latin American press texts, which addresses the lack of specialized corpora for historical and linguistic analysis in this region.

MAPS-KB

MAPS-KB is a million-scale probabilistic simile knowledge base, covering 4.3 million triplets over 0.4 million terms from 70 GB corpora. It is designed for the tasks of simile detection and component extraction. Source:

ACR Appropriateness Criteria Corpus

Dataset Card for the ACR Appropriateness Criteria Corpus This dataset contains chunked guidelines and narratives from the ACR Appropriateness Criteria, an set of societal guidelines from the American College of Radiology (ACR) to help clinicians order appropriate diagnostic imaging studies for patients. The corpus is formatted similarly to the corpuses introduced in MedRAG by Xiong et al. (2024), and can therefore be similarly used for medical Retrieval-Augmented Generation (RAG). Please abide by the ACR Terms and Conditions when using this dataset. ## Dataset Structure Each row of the dataset contains the following fields: { "id": "A unique identifier for each chunk in the corpus", "title": "The title of the main document from which the chunk was extracted", "content": "The content of the chunk", "contents": "The title and content concatenated together into a single field", "ACRID": "The ACR topic identifier" } Of note, the contents field for each row does not exceed a maximum of 2048 characters in our implementation. ## Dataset Curation This dataset was curated by scraping the ACR Appropriateness Criteria website, downloading all the relevant PDF narratives, and using unstructured's open-source tooling to extract the content into chunks. The original source code used for the dataset curation can be found here. This dataset was constructed on July 17, 2024 - no updates to the ACR Appropriateness Criteria after this date are reflected in the most recent iteration. ## Personal and Sensitive Information The contents of this dataset consist of evidence-based guidelines written by a panel of experts for a medical audience. To our knowledge, no personal or sensitive information is contained in this dataset. ## Dataset Card Contact Please contact Michael Yao at michael.yao@pennmedicine.upenn.edu with any questions or comments.

Claim Matching Robustness

  • An evaluation test bed for assessing the robustness of sentence embedding models against user-informed misinformation edits. - A dataset containing perturbed and unperturbed claim pairs used for improving embedding model robustness through knowledge distillation.

CompMix-IR

CompMix-IR Dataset Overview: Characteristics: CompMix-IR is a heterogeneous knowledge retrieval benchmark dataset, featuring four knowledge types (text, knowledge graphs, tables, and infoboxes), 9,400+ QA pairs, and a corpus of 10 million entries. It supports two retrieval scenarios: retrieving across all knowledge types or retrieving specific types based on user instructions. Motivation: It addresses the limitations of existing benchmarks by providing a more comprehensive and realistic dataset that reflects real-world retrieval needs with diverse knowledge sources and user intents. Potential Use Cases: Ideal for developing and evaluating heterogeneous IR models, instruction-aware retrieval systems, and open-domain QA systems. It can also be used for benchmarking, cross-domain IR research, and enhancing the adaptability and robustness of retrieval models.

CRSB

The Official dataset proposed int the paper Context Awareness Gate For Retrieval Augmented Generation

CURE

CURE is a retrieval dataset with a monolingual and two cross-lingual conditions, with splits spanning ten medical domains. Queries in CURE are natural language questions formulated by healthcare providers. They express the information needs of practitioners consulting academic literature in the course of their duties. Queries are available in English, French and Spanish. The corpus is constructed by mining an index of english passages extracted from biomedical academic articles.

DAPFAM

Dataset DAPFAM See the accompanying paper: Ayaou et al., 2025 — “DAPFAM: A Domain‑Aware Patent Retrieval Dataset Aggregated at the Family Level” (arXiv:2506.22141). ## Summary DAPFAM provides 1 247 balanced query patent families and 45 336 target families with forward/backward‑citation relevance labels (≈ 50 K pairs). Each relevant link is explicitly marked in‑domain or out‑of‑domain according to IPC 3‑char overlap, enabling rigorous cross‑domain evaluation. * Full text (title · abstract · claims · description) plus rich metadata for every family. * Multi‑jurisdictional, English‑only text (families may originate in US, JP, EP, CN, …). * Parquet qrel file: qrels_all.parquet. ## Dataset Structure corpus.parquet # 45 336 rows, targets – every original column from the paper queries.parquet # 1 247 rows, queries – same columns + abstract_keywords qrels_all.parquet # (all | in | out) four‑column tables → query_id · relevant_id · relevance_score · domain_rel ## Citation @misc{ayaou2025dapfam, title={DAPFAM: A Domain-Aware Patent Retrieval Dataset Aggregated at the Family Level}, author={Iliass Ayaou and Denis Cavallucci and Hicham Chibane}, year={2025}, eprint={2506.22141}, archivePrefix={arXiv}, primaryClass={cs.CL} } ## Quick Stats * Queries: 1,247 * Corpus (targets): 45,336 * Qrels (all): 49,869 * Qrels (in): 19,736 * Qrels (out): 5,193

Luna-1

This is the supporting dataset for the ECCV 2024 paper "MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain". It contains 5,067 cropped images of craters on the surface of the Moon, generated in the Blender software using publicly available NASA mission data. Also included are 2,161 replicate orbital navigation frames from real-world Lunar Reconnaissance Orbiter (LRO) spacecraft poses with ground-truth bounding box annotations. Luna-1 supports the evaluation of learning-based vision systems for Terrain Relative Navigation (TRN) spacecraft applications.

Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval

The Multi-Eup is a new multilingual benchmark dataset, comprising 22K multilingual documents collected from the European Parliament, spanning 24 languages. This dataset is designed to investigate fairness in a multilingual information retrieval (IR) context to analyze both language and demographic bias in a ranking context. It boasts an authentic multilingual corpus, featuring topics translated into all 24 languages, as well as cross-lingual relevance judgments. Furthermore, it offers rich demographic information associated with its documents, facilitating the study of demographic bias.

PolyNewsParallel

PolyNews is a multilingual parallel dataset containing news titles 833 language pairs, spanning in 64 languages and 17 scripts. PolyNewsParallel aims to provide an easily-accessible, unified and de-duplicated dataset that combines three disparate data sources. It can be used for machine translation or text retrieval in both high-resource and low-resource languages.

ProCIS

A large-scale dataset for proactive document retrieval that consists of over 2.8 million conversations from Reddit.

PTVD

PTVD is a plot-oriented multimodal dataset in the TV domain. It is also the first non-English dataset of its kind. Additionally, PTVD contains more than 26 million bullet screen comments (BSCs), powering large-scale pre-training.

PubMedQA corpus with metadata

PubMedQA-MetaGen: Metadata-Enriched PubMedQA Corpus Dataset Summary PubMedQA-MetaGen is a metadata-enriched version of the PubMedQA biomedical question-answering dataset, created using the MetaGenBlendedRAG enrichment pipeline. The dataset contains both the original and enriched versions of the corpus, enabling direct benchmarking of retrieval-augmented and semantic search approaches in biomedical NLP. Files Provided PubMedQA_original_corpus.json This file contains the original PubMedQA corpus, formatted directly from the official PubMedQA dataset. Each record includes the biomedical question, context (abstract), and answer fields, mirroring the original dataset structure. PubMedQA_corpus_with_metadata.json This file contains the metadata-enriched version, created by processing the original corpus through the MetaGenBlendedRAG pipeline. In addition to the original fields, each entry is augmented with structured metadata—including key concepts, MeSH terms, automatically generated keywords, extracted entities, and LLM-generated summaries—designed to support advanced retrieval and RAG research. How to Use RAG evaluation: Benchmark your retrieval-augmented QA models using the enriched context for higher recall and precision. Semantic Search: Build improved biomedical search engines leveraging topic, entity, and keyword metadata. NLP & LLM Fine-tuning: Use for fine-tuning models that benefit from structured biomedical context. Dataset Structure Each sample contains: Original fields: Question, context (abstract), answer Enriched fields (in PubMedQA_corpus_with_metadata.json only): Key concepts and topics Extracted MeSH terms and UMLS entities Automatically generated keywords Section/type labels LLM-generated summaries/metadata Document identifiers and links Dataset Creation Process Source: Original PubMedQA dataset. Metadata Enrichment: Applied the MetaGenBlendedRAG pipeline (rule-based, NLP, and LLM-driven enrichment). Outputs: Two files—original and enriched—supporting both traditional and metadata-driven research. Intended Use and Limitations For research and educational use in biomedical QA, RAG, semantic retrieval, and metadata enrichment evaluation. Note: Some metadata fields generated by LLMs may vary in quality; users should verify outputs for critical applications.

Spanish Corpus XIX

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

Visual Haystacks (VHs)

Visual Haystacks (VHs) is a "visual-centric" Needle-In-A-Haystack (NIAH) benchmark specifically designed to evaluate the capabilities of Large Multimodal Models (LMMs) in visual retrieval and reasoning over sets of unrelated images. Unlike conventional NIAH challenges that center on text-related retrieval and understanding with limited anecdotal examples, VHs contains a much larger number of examples and focuses on "simple visual tasks", providing a more accurate reflection of LMMs' capabilities when dealing with extensive visual context.

french-legal-cases

--- annotations_creators: - no-annotation language: - fr language_creators: - found license: - cc-by-4.0 multilinguality: - monolingual pretty_name: French Legal Cases Dataset size_categories: - n>1M source_datasets: - la-mousse/INCA-17-01-2025 - la-mousse/JADE-17-01-2025 - la-mousse/CASS-17-01-2025 - la-mousse/CAPP-17-01-2025 task_categories: - text-generation - text-classification task_ids: - language-modeling - entity-linking-classification - fact-checking - intent-classification - multi-label-classification - multi-input-text-classification - natural-language-inference - semantic-similarity-classification - sentiment-classification - topic-classification - sentiment-analysis - named-entity-recognition - parsing - extractive-qa - open-domain-qa - closed-domain-qa - news-articles-summarization - news-articles-headline-generation - dialogue-modeling - dialogue-generation - abstractive-qa - closed-domain-qa - keyword-spotting - semantic-segmentation - tabular-multi-class-classification - tabular-multi-label-classification - document-retrieval - document-question-answering paperswithcode_id: french-legal-cases dataset_info: features: - name: id dtype: string - name: url dtype: string - name: case_number dtype: string - name: contenu dtype: string - name: sommaire dtype: string - name: sommaire_bis dtype: string - name: metadata dtype: string - name: dataset_source dtype: string - name: previous_text dtype: string - name: current_text dtype: string - name: next_text dtype: string - name: triplet_index dtype: int64 configs: - config_name: default data_files: - split: train path: data/train-* --- # Dataset Card for French Legal Cases Dataset ## Dataset Description - Homepage: https://huggingface.co/datasets/la-mousse/combined-fe-caselaw - Repository: https://huggingface.co/datasets/la-mousse/combined-fe-caselaw - Paper: N/A - Point of Contact: [Your Contact Information] ### Dataset Summary This dataset combines French legal cases from multiple sources (INCA, JADE, CASS, CAPP) into a unified format with overlapping text triplets. It includes decisions from various French courts, processed to facilitate natural language processing tasks. ### Supported Tasks and Leaderboards - Tasks: - Text Generation - Legal Document Analysis - Text Classification - Language Modeling ### Languages The dataset is monolingual (French). ## Dataset Structure ### Data Instances Each instance contains: - Document identifiers (id, url, case_number) - Original text content (contenu) - Summaries (sommaire, sommaire_bis) - Metadata (JSON formatted string) - Text triplets (previous_text, current_text, next_text) - Position tracking (triplet_index, window_index) Example: python { 'id': 'CASS12345', 'url': 'https://...', 'case_number': '12-34567', 'contenu': 'Full text...', 'sommaire': 'Summary...', 'sommaire_bis': 'Additional summary...', 'metadata': '{"date_decision": "2023-01-01", ...}', 'dataset_source': 'CASS', 'previous_text': 'Previous chunk...', 'current_text': 'Current chunk...', 'next_text': 'Next chunk...', 'triplet_index': 0, 'window_index': 0 } ### Data Fields - id: Unique identifier - url: Source URL - case_number: Case reference number - contenu: Full text content - sommaire: Primary summary - sommaire_bis: Secondary summary - metadata: JSON string containing additional metadata - dataset_source: Origin dataset (INCA/JADE/CASS/CAPP) - previous_text: Previous text chunk - current_text: Current text chunk - next_text: Next text chunk - triplet_index: Position in sequence of triplets - window_index: Window number for long texts ### Data Splits - Training split only ## Dataset Creation ### Curation Rationale This dataset was created to provide a standardized format for French legal texts, with overlapping text chunks suitable for various NLP tasks. ### Source Data #### Initial Data Collection and Normalization - INCA: Court of Cassation decisions - JADE: Administrative court decisions - CASS: Court of Cassation decisions - CAPP: Court of Appeal decisions ### Preprocessing - Text chunking with 230-token chunks and 30-token overlap - Sliding window approach for long texts - Metadata preservation and standardization - Token count verification - JSON formatting for metadata ### Quality Control - Token length verification - Chunk coherence checks - Metadata validation - Error logging and handling ## Considerations for Using the Data ### Social Impact of Dataset This dataset aims to improve access to and understanding of French legal decisions, potentially benefiting legal research and analysis. ### Discussion of Biases The dataset may reflect inherent biases in the French legal system and case selection/publication processes. ### Other Known Limitations - Limited to published decisions - Varying detail levels across sources - Potential OCR errors in source texts ## Additional Information ### Dataset Curators La-Mousse ### Licensing Information CC-BY-4.0 ### Citation Information @misc{french-legal-cases-2024, title={French Legal Cases Dataset}, author={[Your Name]}, year={2024}, publisher={HuggingFace}, url={https://huggingface.co/datasets/la-mousse/combined-fr-caselaw} } ### Contributions Thanks to @huggingface for the dataset hosting and infrastructure.

Multimodal Needle in a Haystack (MMNeedle)

Multimodal Large Language Models (MLLMs) have shown significant promise in various applications, leading to broad interest from researchers and practitioners alike. However, a comprehensive evaluation of their long-context capabilities remains underexplored. To address these gaps, we introduce the MultiModal Needle-in-a-haystack (MMNeedle) benchmark, specifically designed to assess the long-context capabilities of MLLMs. Besides multi-image input, we employ image stitching to further increase the input context length, and develop a protocol to automatically generate labels for sub-image level retrieval. Essentially, MMNeedle evaluates MLLMs by stress-testing their capability to locate a target sub-image (needle) within a set of images (haystack) based on textual instructions and descriptions of image contents. This setup necessitates an advanced understanding of extensive visual contexts and effective information retrieval within long-context image inputs. With this benchmark, we evaluate state-of-the-art MLLMs, encompassing both API-based and open-source models. The findings reveal that GPT-4o consistently surpasses other models in long-context scenarios, but suffers from hallucination problems in negative samples, i.e., when needles are not in the haystacks. Our comprehensive long-context evaluation of MLLMs also sheds lights on the considerable performance gap between API-based and open-source models.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.