Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing
원문 초록
We describe models focused at the understudied problem of translating between monolingual and code-mixed language pairs. More specifically, we offer a wide range of models that convert monolingual English text into Hinglish (code-mixed Hindi and English). Given the recent success of pretrained language models, we also test the utility of two recent Transformer-based encoder-decoder models (i.e., mT5 and mBART) on the task finding both to work well. Given the paucity of training data for code-mixing, we also propose a dependency-free method for generating code-mixed texts from bilingual distributed representations that we exploit for improving language model performance. In particular, armed with this additional data, we adopt a curriculum learning approach where we first finetune the language models on synthetic data then on gold code-mixed data. We find that, although simple, our synthetic code-mixing method is competitive with (and in some cases is even superior to) several standard methods (backtranslation, method based on equivalence constraint theory) under a diverse set of conditions. Our work shows that the mT5 model, finetuned following the curriculum learning procedure, achieves best translation performance (12.67 BLEU). Our models place first in the overall ranking of the English-Hinglish official shared task.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Effective Deep Learning Models for Automatic Diacritization of Arabic Text
Making Language Model a Hierarchical Classifier and Generator
CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
Pre-Trained LLM is a Semantic-Aware and Generalizable Segmentation Booster
Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings
Speak2Sign3D: A Multi-modal Pipeline for English Speech to American Sign Language Animation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.