Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation
원문 초록
Out-of-Vocabulary (OOV) is a problem for Neural Machine Translation (NMT). OOV refers to words with a low occurrence in the training data, or to those that are absent from the training data. To alleviate this, word or phrase-based Data Augmentation (DA) techniques have been used. However, existing DA techniques have addressed only one of these OOV types and limit to considering either syntactic constraints or semantic constraints. We present a word and phrase replacement-based DA technique that consider both types of OOV, by augmenting (1) rare words in the existing parallel corpus, and (2) new words from a bilingual dictionary. During augmentation, we consider both syntactic and semantic properties of the words to guarantee fluency in the synthetic sentences. This technique was experimented with low resource Sinhala-English language pair. We observe with only semantic constraints in the DA, the results are comparable with the scores obtained considering syntactic constraints, and is favourable for low-resourced languages that lacks linguistic tool support. Additionally, results can be further improved by considering both syntactic and semantic constraints.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT
Building a Functional Machine Translation Corpus for Kpelle
In-Domain African Languages Translation Using LLMs and Multi-armed Bandits
THOR-MoE: Hierarchical Task-Guided and Context-Responsive Routing for Neural Machine Translation
Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.