Similarity Based Label Smoothing For Dialogue Generation
원문 초록
Generative neural conversational systems are generally trained with the objective of minimizing the entropy loss between the training "hard" targets and the predicted logits. Often, performance gains and improved generalization can be achieved by using regularization techniques like label smoothing, which converts the training "hard" targets to "soft" targets. However, label smoothing enforces a data independent uniform distribution on the incorrect training targets, which leads to an incorrect assumption of equi-probable incorrect targets for each correct target. In this paper we propose and experiment with incorporating data dependent word similarity based weighing methods to transforms the uniform distribution of the incorrect target probabilities in label smoothing, to a more natural distribution based on semantics. We introduce hyperparameters to control the incorrect target distribution, and report significant performance gains over networks trained using standard label smoothing based loss, on two standard open domain dialogue corpora.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching
SDialog: A Python Toolkit for Synthetic Dialogue Generation and Analysis
Enhancing Medical Dialogue Generation through Knowledge Refinement and Dynamic Prompt Adjustment
Efficient Tuning of Large Language Models for Knowledge-Grounded Dialogue Generation
VisTai: Benchmarking Vision-Language Models for Traditional Chinese in Taiwan
SAGE: Steering and Refining Dialog Generation with State-Action Augmentation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.