Know When To Stop: A Study of Semantic Drift in Text Generation
원문 초록
In this work, we explicitly show that modern LLMs tend to generate correct facts first, then "drift away" and generate incorrect facts later: this was occasionally observed but never properly measured. We develop a semantic drift score that measures the degree of separation between correct and incorrect facts in generated texts and confirm our hypothesis when generating Wikipedia-style biographies. This correct-then-incorrect generation pattern suggests that factual accuracy can be improved by knowing when to stop generation. Therefore, we explore the trade-off between information quantity and factual accuracy for several early stopping methods and manage to improve factuality by a large margin. We further show that reranking with semantic similarity can further improve these results, both compared to the baseline and when combined with early stopping. Finally, we try calling external API to bring the model back to the right generation path, but do not get positive results. Overall, our methods generalize and can be applied to any long-form text generation to produce more reliable information, by balancing trade-offs between factual accuracy, information quantity and computational cost.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation
Temporally-Grounded Language Generation: A Benchmark for Real-Time Vision-Language Models
Concept-Level Explainability for Auditing & Steering LLM Responses
SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
InsertRank: LLMs can reason over BM25 scores to Improve Listwise Reranking
FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.