Language Modelling
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 6
- 평가표
- 55
- 보존된 평가 기록
- 466
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
코드
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
https://github.com/lucidrains/FLASH-pytorch
Primer: Searching for Efficient Transformers for Language Modeling
https://github.com/lucidrains/PaLM-jax
https://github.com/lucidrains/PaLM-pytorch
https://github.com/lucidrains/RETRO-pytorch
Improving language models by retrieving from trillions of tokens
https://github.com/lucidrains/audiolm-pytorch
https://github.com/lucidrains/block-recurrent-transformer-pytorch
https://github.com/lucidrains/coco-lm-pytorch
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
https://github.com/lucidrains/compressive-transformer-pytorch
https://github.com/lucidrains/conformer
Conformer: Convolution-augmented Transformer for Speech Recognition
https://github.com/lucidrains/electra-pytorch
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
https://github.com/lucidrains/ema-pytorch
https://github.com/lucidrains/feedback-transformer-pytorch
Addressing Some Limitations of Transformers with Feedback Memory
https://github.com/lucidrains/flamingo-pytorch
https://github.com/lucidrains/flash-cosine-sim-attention
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
https://github.com/lucidrains/gated-state-spaces-pytorch
https://github.com/lucidrains/h-transformer-1d
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences
https://github.com/lucidrains/hourglass-transformer-pytorch
Hierarchical Transformers Are More Efficient Language Models
https://github.com/lucidrains/infini-transformer-pytorch
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
https://github.com/lucidrains/linear-attention-transformer
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
https://github.com/lucidrains/lion-pytorch
https://github.com/lucidrains/local-attention
Efficient Content-Based Sparse Attention with Routing Transformers
https://github.com/lucidrains/long-short-transformer
Long-Short Transformer: Efficient Transformers for Language and Vision
https://github.com/lucidrains/magvit2-pytorch
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
https://github.com/lucidrains/marge-pytorch
https://github.com/lucidrains/mlm-pytorch
https://github.com/lucidrains/muse-pytorch
Muse: Text-To-Image Generation via Masked Generative Transformers
https://github.com/lucidrains/omninet-pytorch
https://github.com/lucidrains/perceiver-ar-pytorch
General-purpose, long-context autoregressive modeling with Perceiver AR
https://github.com/lucidrains/performer-pytorch
https://github.com/lucidrains/product-key-memory
https://github.com/lucidrains/progen
https://github.com/lucidrains/progen-jax
https://github.com/lucidrains/reformer-pytorch
https://github.com/lucidrains/rotary-embedding-torch
https://github.com/lucidrains/routing-transformer
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.