Connecting the Dots Between MLE and RL for Sequence Prediction
원문 초록
Sequence prediction models can be learned from example sequences with a variety of training algorithms. Maximum likelihood learning is simple and efficient, yet can suffer from compounding error at test time. Reinforcement learning such as policy gradient addresses the issue but can have prohibitively poor exploration efficiency. A rich set of other algorithms such as RAML, SPG, and data noising, have also been developed from different perspectives. This paper establishes a formal connection between these algorithms. We present a generalized entropy regularized policy optimization formulation, and show that the apparently distinct algorithms can all be reformulated as special instances of the framework, with the only difference being the configurations of a reward function and a couple of hyperparameters. The unified interpretation offers a systematic view of the varying properties of exploration and learning efficiency. Besides, inspired from the framework, we present a new algorithm that dynamically interpolates among the family of algorithms for scheduled sequence model learning. Experiments on machine translation, text summarization, and game imitation learning demonstrate the superiority of the proposed algorithm.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
Gender Bias in English-to-Greek Machine Translation
TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaboration
Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations
Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation
Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement
토론
이 논문과 연결된 포럼 글이 아직 없습니다.