arXiv:2407.13657
FuLG: 150B Romanian Corpus for Language Model Pretraining
원문 초록
Research in the field of language models is rapidly evolving, with many open models being released to the public. Openly available pretraining corpora usually focus on only a handful of languages, with many others either missing completely or extremely underrepresented. In this report, we introduce FuLG, a hundred-fifty-billion-token Romanian corpus extracted from CommonCrawl. We present our methodology for filtering FuLG and compare it via ablation studies against existing Romanian corpora.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Making Language Model a Hierarchical Classifier and Generator
CoLMbo: Speaker Language Model for Descriptive Profiling
Data Efficacy for Language Model Training
Enterprise Large Language Model Evaluation Benchmark
Research on Model Parallelism and Data Parallelism Optimization Methods in Large Language Model-Based Recommendation Systems
Finance Language Model Evaluation (FLaME)
토론
이 논문과 연결된 포럼 글이 아직 없습니다.