Bilevel Scheduled Sampling for Dialogue Generation
원문 초록
Exposure bias poses a common challenge in numerous natural language processing tasks, particularly in the dialog generation. In response to this issue, researchers have devised various techniques, among which scheduled sampling has proven to be an effective method for mitigating exposure bias. However, the existing state-of-the-art scheduled sampling methods solely consider the current sampling words' quality for threshold truncation sampling, which overlooks the importance of sentence-level information and the method of threshold truncation warrants further discussion. In this paper, we propose a bilevel scheduled sampling model that takes the sentence-level information into account and incorporates it with word-level quality. To enhance sampling diversity and improve the model's adaptability, we propose a smooth function that maps the combined result of sentence-level and word-level information to an appropriate range, and employ probabilistic sampling based on the mapped values instead of threshold truncation. Experiments conducted on the DailyDialog and PersonaChat datasets demonstrate the effectiveness of our proposed methods, which significantly alleviate the exposure bias problem and outperform state-of-the-art scheduled sampling methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matching
When Harry Meets Superman: The Role of The Interlocutor in Persona-Based Dialogue Generation
Coherent Visual Storytelling via Parallel Top-Down Visual and Topic Attention
Imitation Learning for Sentence Generation with Dilated Convolutions Using Adversarial Training
Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression
Mitigating Object Hallucinations via Sentence-Level Early Intervention
토론
이 논문과 연결된 포럼 글이 아직 없습니다.