Normalization of Transliterated Words in Code-Mixed Data Using Seq2Seq Model & Levenshtein Distance
원문 초록
Building tools for code-mixed data is rapidly gaining popularity in the NLP research community as such data is exponentially rising on social media. Working with code-mixed data contains several challenges, especially due to grammatical inconsistencies and spelling variations in addition to all the previous known challenges for social media scenarios. In this article, we present a novel architecture focusing on normalizing phonetic typing variations, which is commonly seen in code-mixed data. One of the main features of our architecture is that in addition to normalizing, it can also be utilized for back-transliteration and word identification in some cases. Our model achieved an accuracy of 90.27% on the test data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
ParsiPy: NLP Toolkit for Historical Persian Texts in Python
Sinhala Transliteration: A Comparative Analysis Between Rule-based and Seq2Seq Approaches
Romanized to Native Malayalam Script Transliteration Using an Encoder-Decoder Framework
GR-NLP-TOOLKIT: An Open-Source NLP Toolkit for Modern Greek
How Transliterations Improve Crosslingual Alignment
Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
토론
이 논문과 연결된 포럼 글이 아직 없습니다.