Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters
원문 초록
Adapter layers are lightweight, learnable units inserted between transformer layers. Recent work explores using such layers for neural machine translation (NMT), to adapt pre-trained models to new domains or language pairs, training only a small set of parameters for each new setting (language pair or domain). In this work we study the compositionality of language and domain adapters in the context of Machine Translation. We aim to study, 1) parameter-efficient adaptation to multiple domains and languages simultaneously (full-resource scenario) and 2) cross-lingual transfer in domains where parallel data is unavailable for certain language pairs (partial-resource scenario). We find that in the partial resource scenario a naive combination of domain-specific and language-specific adapters often results in `catastrophic forgetting' of the missing languages. We study other ways to combine the adapters to alleviate this issue and maximize cross-lingual transfer. With our best adapter combinations, we obtain improvements of 3-4 BLEU on average for source languages that do not have in-domain data. For target languages without in-domain data, we achieve a similar improvement by combining adapters with back-translation. Supplementary material is available at https://tinyurl.com/r66stbxj
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
In-Domain African Languages Translation Using LLMs and Multi-armed Bandits
TULUN: Transparent and Adaptable Low-resource Machine Translation
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT
THOR-MoE: Hierarchical Task-Guided and Context-Responsive Routing for Neural Machine Translation
Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation
CoCoNuT: Combining Context-Aware Neural Translation Models using Ensemble for Program Repair
토론
이 논문과 연결된 포럼 글이 아직 없습니다.