Language Modelling
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 6
- 평가표
- 55
- 보존된 평가 기록
- 466
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
코드
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
https://github.com/MindSpore-paper-code-2/code3/tree/main/ViG
https://github.com/MindSpore-paper-code-2/code3/tree/main/t2t-vit
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
https://github.com/MindSpore-paper-code-2/code399/tree/main/Bert
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
https://github.com/MindSpore-paper-code-2/code399/tree/main/Bert_thor
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
https://github.com/MindSpore-paper-code-2/code399/tree/main/GPT
https://github.com/MindSpore-paper-code-2/code399/tree/main/Pangu_alpha
PanGu-Coder: Program Synthesis with Function-Level Language Modeling
https://github.com/MindSpore-paper-code-2/code399/tree/main/jasper
https://github.com/MindSpore-paper-code-2/code400/tree/main/DeepText
https://github.com/MindSpore-paper-code-3/code10/tree/main/guided_anchoring
https://github.com/MindSpore-paper-code-3/code9/tree/main/DeepText
https://github.com/MindSpore-paper-code-3/code9/tree/main/cpm
CPM: A Large-scale Generative Chinese Pre-trained Language Model
https://github.com/MindSpore-paper-code-3/code9/tree/main/emotect
ERNIE: Enhanced Representation through Knowledge Integration
https://github.com/MindSpore-scientific-2/code-10/tree/main/QuantExplainNLP_AlgoAnalysis
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
https://github.com/MindSpore-scientific-2/code-10/tree/main/token_learner
LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings
https://github.com/MindSpore-scientific-2/code-11/tree/main/token_learner
LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings
https://github.com/MindSpore-scientific-2/code-12/tree/main/gated-word-character-recurrent-language-model
https://github.com/MindSpore-scientific-2/code-12/tree/main/isr
ISR-DPO: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective DPO
https://github.com/MindSpore-scientific-2/code-14/tree/main/gpt_neox_japanese
https://github.com/MindSpore-scientific-2/code-14/tree/main/gptj
https://github.com/MindSpore-scientific-2/code-14/tree/main/hubert
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
https://github.com/MindSpore-scientific-2/code-14/tree/main/jamba
https://github.com/MindSpore-scientific-2/code-14/tree/main/layoutlmv2
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
https://github.com/MindSpore-scientific-2/code-14/tree/main/layoutlmv3
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
https://github.com/MindSpore-scientific-2/code-14/tree/main/longt5
LongT5: Efficient Text-To-Text Transformer for Long Sequences
https://github.com/MindSpore-scientific-2/code-2/tree/main/token_learner
LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings
https://github.com/MindSpore-scientific-2/code-3/tree/main/TSGD
https://github.com/MindSpore-scientific-2/code-4/tree/main/Hyena-A-Convolutional-Neural-Network-for-Modelling-Sentences
Hyena Hierarchy: Towards Larger Convolutional Language Models
https://github.com/MindSpore-scientific-2/code-5/tree/main/Deep-Gradient-Compression
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
https://github.com/MindSpore-scientific-2/code-5/tree/main/Knowing-When-to-Look-Adaptive-Attention
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
https://github.com/MindSpore-scientific-2/code-8/tree/main/ZoneoutRNN
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
https://github.com/MindSpore-scientific-2/code-8/tree/main/shallow-rnns
Shallow Fusion of Weighted Finite-State Transducer and Language Model for Text Normalization
https://github.com/MindSpore-scientific-2/code-8/tree/main/text-to-music
https://github.com/MindSpore-scientific-2/code-9/tree/main/Deep-Gradient-Compression
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
https://github.com/MindSpore-scientific-2/code-9/tree/main/Knowing-When-to-Look-Adaptive-Attention
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
https://github.com/MindSpore-scientific/code-1/tree/main/Knowing-When-to-Look-Adaptive-Attention
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.