https://github.com/PKU-YuanGroup/MoE-LLaVA
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
ERNIE: Enhanced Representation through Knowledge Integration
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
XLNet: Generalized Autoregressive Pretraining for Language Understanding
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
CTRL: A Conditional Transformer Language Model for Controllable Generation
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
MPNet: Masked and Permuted Pre-training for Language Understanding
TinyBERT: Distilling BERT for Natural Language Understanding
ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network
First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.