A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
원문 초록
We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then compare the performance of OSCAR-based and Wikipedia-based ELMo embeddings for these languages on the part-of-speech tagging and parsing tasks. We show that, despite the noise in the Common-Crawl-based OSCAR data, embeddings trained on OSCAR perform much better than monolingual embeddings trained on Wikipedia. They actually equal or improve the current state of the art in tagging and parsing for all five languages. In particular, they also improve over multilingual Wikipedia-based contextual embeddings (multilingual BERT), which almost always constitutes the previous state of the art, thereby showing that the benefit of a larger, more diverse corpus surpasses the cross-lingual benefit of multilingual embedding architectures.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
A Comparative Analysis of Static Word Embeddings for Hungarian
VNLP: Turkish NLP Package
myNER: Contextualized Burmese Named Entity Recognition with Bidirectional LSTM and fastText Embeddings via Joint Training with POS Tagging
Exploring transfer learning for Deep NLP systems on rarely annotated languages
Bit Cipher -- A Simple yet Powerful Word Representation System that Integrates Efficiently with Language Models
Explainable Identification of Hate Speech towards Islam using Graph Neural Networks
토론
이 논문과 연결된 포럼 글이 아직 없습니다.