arXiv:2312.15576

Reducing LLM Hallucinations using Epistemic Neural Networks

Shreyas Verma, Kien Tran, Yusuf Ali, Guangyu Min

제출 2023-12-25

원문 초록

Reducing and detecting hallucinations in large language models is an open research problem. In this project, we attempt to leverage recent advances in the field of uncertainty estimation to reduce hallucinations in frozen large language models. Epistemic neural networks have recently been proposed to improve output joint distributions for large pre-trained models. ENNs are small networks attached to large, frozen models to improve the model's joint distributions and uncertainty estimates. In this work, we train an epistemic neural network on top of the Llama-2 7B model combined with a contrastive decoding feature enhancement technique. We are the first to train an ENN for the next token prediction task and explore the efficacy of this method in reducing hallucinations on the TruthfulQA dataset. In essence, we provide a method that leverages a pre-trained model's latent embeddings to reduce hallucinations.

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

arXiv:2505.12182공통 태그 TruthfulQA공식 코드 · Papers with Code

Truth Neurons

토론

이 논문과 연결된 포럼 글이 아직 없습니다.