Turkish PoS Tagging by Reducing Sparsity with Morpheme Tags in Small Datasets
원문 초록
Sparsity is one of the major problems in natural language processing. The problem becomes even more severe in agglutinating languages that are highly prone to be inflected. We deal with sparsity in Turkish by adopting morphological features for part-of-speech tagging. We learn inflectional and derivational morpheme tags in Turkish by using conditional random fields (CRF) and we employ the morpheme tags in part-of-speech (PoS) tagging by using hidden Markov models (HMMs) to mitigate sparsity. Results show that using morpheme tags in PoS tagging helps alleviate the sparsity in emission probabilities. Our model outperforms other hidden Markov model based PoS tagging models for small training datasets in Turkish. We obtain an accuracy of 94.1% in morpheme tagging and 89.2% in PoS tagging on a 5K training dataset.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
A Comparative Analysis of Static Word Embeddings for Hungarian
POS-tagging to highlight the skeletal structure of sentences
How Does A Text Preprocessing Pipeline Affect Ontology Syntactic Matching?
J2N -- Nominal Adjective Identification and its Application
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
Multi-Task Learning for Front-End Text Processing in TTS
토론
이 논문과 연결된 포럼 글이 아직 없습니다.