A Statistical Framework for Data-dependent Retrieval-Augmented Models
원문 초록
Modern ML systems increasingly augment input instances with additional relevant information to enhance final prediction. Despite growing interest in such retrieval-augmented models, their fundamental properties and training are not well understood. We propose a statistical framework to study such models with two components: 1) a {\em retriever} to identify the relevant information out of a large corpus via a data-dependent metric; and 2) a {\em predictor} that consumes the input instances along with the retrieved information to make the final predictions. We present a principled method for end-to-end training of both components and draw connections with various training approaches in the literature. Furthermore, we establish excess risk bounds for retrieval-augmented models while delineating the contributions of both retriever and predictor towards the model performance. We validate the utility of our proposed training methods along with the key takeaways from our statistical analysis on open domain question answering task where retrieval augmentation is important.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoning
Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-$k$
Answer Complex Questions: Path Ranker Is All You Need
From Roots to Rewards: Dynamic Tree Reasoning with RL
Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality
SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinement
토론
이 논문과 연결된 포럼 글이 아직 없습니다.