Test perplexity: 16.61
Papers with Code · CC BY-SA 4.0
Language Modelling
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
논문
Test perplexity: 16.9Number of params: 355M
Test perplexity: 18.5
Validation perplexity: 16.89Test perplexity: 17.56Number of params: 247M
Validation perplexity: 16.54Test perplexity: 17.18
Validation perplexity: 16.4Test perplexity: 17.1Number of params: 234M
Validation perplexity: 17.97Test perplexity: 18.70Number of params: 247M
Validation perplexity: 17.5Test perplexity: 18.2Number of params: 139M
Validation perplexity: 19.7Test perplexity: 20.6Number of params: 133M
Validation perplexity: 16.0Test perplexity: 17.1
Test perplexity: 18.0
Validation perplexity: 16.3Test perplexity: 17.0Number of params: 257M
Test perplexity: 21.28Number of params: 249M
Validation perplexity: 19Test perplexity: 19.6
Test perplexity: 18.4
Test perplexity: 17.4Number of params: 257M
Test perplexity: 17.96Number of params: 247M
Test perplexity: 17.48Number of params: 1542M
Validation perplexity: 19.6Test perplexity: 20.4Number of params: 395M
Test perplexity: 18.07Number of params: 252M
Test perplexity: 18.4
Test perplexity: 17.1Number of params: 257M
Test perplexity: 20.39Number of params: 96M
Validation perplexity: 18.2Test perplexity: 18.3Number of params: 257M