Controlled Automatic Task-Specific Synthetic Data Generation for Hallucination Detection
원문 초록
We present a novel approach to automatically generate non-trivial task-specific synthetic datasets for hallucination detection. Our approach features a two-step generation-selection pipeline, using hallucination pattern guidance and a language style alignment during generation. Hallucination pattern guidance leverages the most important task-specific hallucination patterns while language style alignment aligns the style of the synthetic dataset with benchmark text. To obtain robust supervised detectors from synthetic datasets, we also adopt a data mixture strategy to improve performance robustness and generalization. Our results on three datasets show that our generated hallucination text is more closely aligned with non-hallucinated text versus baselines, to train hallucination detectors with better generalization. Our hallucination detectors trained on synthetic datasets outperform in-context-learning (ICL)-based detectors by a large margin of 32%. Our extensive experiments confirm the benefits of our approach with cross-task and cross-generator generalization. Our data-mixture-based training further improves the generalization and robustness of hallucination detection.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
BEAR: BGP Event Analysis and Reporting
DocIE@XLLM25: In-Context Learning for Information Extraction using Fully Synthetic Demonstrations
Does Prompt Design Impact Quality of Data Imputation by LLMs?
The Prompt is Mightier than the Example
Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation
Reinforcing Question Answering Agents with Minimalist Policy Gradient Optimization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.