Understanding and Predicting Characteristics of Test Collections in Information Retrieval
원문 초록
Research community evaluations in information retrieval, such as NIST's Text REtrieval Conference (TREC), build reusable test collections by pooling document rankings submitted by many teams. Naturally, the quality of the resulting test collection thus greatly depends on the number of participating teams and the quality of their submitted runs. In this work, we investigate: i) how the number of participants, coupled with other factors, affects the quality of a test collection; and ii) whether the quality of a test collection can be inferred prior to collecting relevance judgments from human assessors. Experiments conducted on six TREC collections illustrate how the number of teams interacts with various other factors to influence the resulting quality of test collections. We also show that the reusability of a test collection can be predicted with high accuracy when the same document collection is used for successive years in an evaluation campaign, as is common in TREC.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Dynamic Modality Interaction Modeling for Image-Text Retrieval
JointRank: Rank Large Set with Single Pass
From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.