Regularization techniques for fine-tuning in neural machine translation
원문 초록
We investigate techniques for supervised domain adaptation for neural machine translation where an existing model trained on a large out-of-domain dataset is adapted to a small in-domain dataset. In this scenario, overfitting is a major challenge. We investigate a number of techniques to reduce overfitting and improve transfer learning, including regularization techniques such as dropout and L2-regularization towards an out-of-domain prior. In addition, we introduce tuneout, a novel regularization technique inspired by dropout. We apply these techniques, alone and in combination, to neural machine translation, obtaining improvements on IWSLT datasets for English->German and English->Russian. We also investigate the amounts of in-domain training data needed for domain adaptation in NMT, and find a logarithmic relationship between the amount of training data and gain in BLEU score.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Cross-Domain Conditional Diffusion Models for Time Series Imputation
BotTrans: A Multi-Source Graph Domain Adaptation Approach for Social Bot Detection
Flowing Datasets with Wasserstein over Wasserstein Gradient Flows
Unleashing the Power of Intermediate Domains for Mixed Domain Semi-Supervised Medical Image Segmentation
The Bayesian Approach to Continual Learning: An Overview
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT
토론
이 논문과 연결된 포럼 글이 아직 없습니다.