Homograph Disambiguation Through Selective Diacritic Restoration
원문 초록
Lexical ambiguity, a challenging phenomenon in all natural languages, is particularly prevalent for languages with diacritics that tend to be omitted in writing, such as Arabic. Omitting diacritics leads to an increase in the number of homographs: different words with the same spelling. Diacritic restoration could theoretically help disambiguate these words, but in practice, the increase in overall sparsity leads to performance degradation in NLP applications. In this paper, we propose approaches for automatically marking a subset of words for diacritic restoration, which leads to selective homograph disambiguation. Compared to full or no diacritic restoration, these approaches yield selectively-diacritized datasets that balance sparsity and lexical disambiguation. We evaluate the various selection strategies extrinsically on several downstream applications: neural machine translation, part-of-speech tagging, and semantic textual similarity. Our experiments on Arabic show promising results, where our devised strategies on selective diacritization lead to a more balanced and consistent performance in downstream applications.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Intrinsic vs. Extrinsic Evaluation of Czech Sentence Embeddings: Semantic Relevance Doesn't Help with MT Evaluation
Effective Deep Learning Models for Automatic Diacritization of Arabic Text
CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
Gender Bias in English-to-Greek Machine Translation
TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaboration
Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations
토론
이 논문과 연결된 포럼 글이 아직 없습니다.