Language Modelling
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 6
- 평가표
- 55
- 보존된 평가 기록
- 466
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
코드
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
https://github.com/expz/annotated-hyena
Hyena Hierarchy: Towards Larger Convolutional Language Models
https://github.com/expz/quiet-star
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
https://github.com/eyalbetzalel/pytorch-generative
https://github.com/eyalbetzalel/pytorch-generative-v2
https://github.com/eyalbetzalel/pytorch-generative-v6
https://github.com/f1yfisher/drivedreamer2
DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation
https://github.com/f51980280/ENAS-Implement
https://github.com/fabianwinter93/JAX/tree/main/FastFeedForwardNetwork
https://github.com/fabianwinter93/JAX/tree/main/GateLoop
GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling
https://github.com/fabianwinter93/JAX/tree/main/QLSTM
The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute
https://github.com/fabio-deep/ReZero-ResNet
https://github.com/facebook/MazeBase
https://github.com/facebook/bAbI-tasks
https://github.com/facebookarchive/MazeBase
https://github.com/facebookarchive/SCRNNs
https://github.com/facebookarchive/babi-tasks
https://github.com/facebookresearch/ActivityNet-Entities
Learning to Generate Grounded Visual Captions without Localization Supervision
https://github.com/facebookresearch/CodeGen
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
https://github.com/facebookresearch/DisCo
Mask-Predict: Parallel Decoding of Conditional Masked Language Models
https://github.com/facebookresearch/EgoTV
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
https://github.com/facebookresearch/InferSent
https://github.com/facebookresearch/MLQA
https://github.com/facebookresearch/ParlAI
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
https://github.com/facebookresearch/SentEval
https://github.com/facebookresearch/UnsupervisedMT
https://github.com/facebookresearch/anli
XLNet: Generalized Autoregressive Pretraining for Language Understanding
https://github.com/facebookresearch/audiocraft
https://github.com/facebookresearch/code-prediction-transformer
Character-Level Language Modeling with Deeper Self-Attention
https://github.com/facebookresearch/data2vec_vision
Unified Language Model Pre-training for Natural Language Understanding and Generation
https://github.com/facebookresearch/doc-storygen-v2
OpenAssistant Conversations -- Democratizing Large Language Model Alignment
https://github.com/facebookresearch/fairscale
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
https://github.com/facebookresearch/fairseq
https://github.com/facebookresearch/fairseq/tree/ust/examples/w2vbert
https://github.com/facebookresearch/fairseq2
https://github.com/facebookresearch/llama/blob/6c7fe276574e78057f917549435a2554000a876d/llama/model.py
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.