Test perplexity: 24.22
Papers with Code · CC BY-SA 4.0
Language Modelling
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
논문
Test perplexity: 23.2Number of params: 110M
Validation perplexity: 29.0Test perplexity: 29.2
Test perplexity: 22.05Number of params: 774M
Validation perplexity: 22Test perplexity: 23.5
Validation perplexity: 21.4Test perplexity: 22.4Number of params: 44M
Validation perplexity: 31.92Test perplexity: 32.85
Validation perplexity: 32.0Test perplexity: 33.0Number of params: 151M
Test perplexity: 24.14Number of params: 99M
Test perplexity: 25.81Number of params: 144.4M
Test perplexity: 23.7Number of params: 125M
Validation perplexity: 27.2Test perplexity: 28.0
Validation perplexity: 21.87Test perplexity: 22.91Number of params: 122M
Test perplexity: 22.7
Test perplexity: 31.0
Test perplexity: 26.0
Validation perplexity: 30.8Test perplexity: 31.6
Test perplexity: 26.8
Validation perplexity: 24.1Test perplexity: 25.2Number of params: 148M
Test perplexity: 23.3Number of params: 240M
Validation perplexity: 23.1Test perplexity: 24.0Number of params: 151M
Test perplexity: 25.6
Test perplexity: 29.19