Oh My Mistake!: Toward Realistic Dialogue State Tracking including Turnback Utterances
원문 초록
The primary purpose of dialogue state tracking (DST), a critical component of an end-to-end conversational system, is to build a model that responds well to real-world situations. Although we often change our minds from time to time during ordinary conversations, current benchmark datasets do not adequately reflect such occurrences and instead consist of over-simplified conversations, in which no one changes their mind during a conversation. As the main question inspiring the present study, "Are current benchmark datasets sufficiently diverse to handle casual conversations in which one changes their mind after a certain topic is over?" We found that the answer is "No" because DST models cannot refer to previous user preferences when template-based turnback utterances are injected into the dataset. Even in the the simplest mind-changing (turnback) scenario, the performance of DST models significantly degenerated. However, we found that this performance degeneration can be recovered when the turnback scenarios are explicitly designed in the training set, implying that the problem is not with the DST models but rather with the construction of the benchmark dataset.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling
Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot
A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understanding
Confidence Estimation for LLM-Based Dialogue State Tracking
Continual Dialogue State Tracking via Reason-of-Select Distillation
TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.