Open Domain Question Answering with Conflicting Contexts
원문 초록
Open domain question answering systems frequently rely on information retrieved from large collections of text (such as the Web) to answer questions. However, such collections of text often contain conflicting information, and indiscriminately depending on this information may result in untruthful and inaccurate answers. To understand the gravity of this problem, we collect a human-annotated dataset, Question Answering with Conflicting Contexts (QACC), and find that as much as 25% of unambiguous, open domain questions can lead to conflicting contexts when retrieved using Google Search. We evaluate and benchmark three powerful Large Language Models (LLMs) with our dataset QACC and demonstrate their limitations in effectively addressing questions with conflicting information. To explore how humans reason through conflicting contexts, we request our annotators to provide explanations for their selections of correct answers. We demonstrate that by finetuning LLMs to explain their answers, we can introduce richer information into their training that guide them through the process of reasoning with conflicting contexts.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoning
ECoRAG: Evidentiality-guided Compression for Long Context RAG
GenKI: Enhancing Open-Domain Question Answering with Knowledge Integration and Controllable Generation in Large Language Models
Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-$k$
Answer Complex Questions: Path Ranker Is All You Need
From Roots to Rewards: Dynamic Tree Reasoning with RL
토론
이 논문과 연결된 포럼 글이 아직 없습니다.