https://github.com/hit-scir/chinese-mixtral-8x7b
Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.
Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.
Source: Wikipedia
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training
A LLM-Based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
Waffling around for Performance: Visual Classification with Random Words and Broad Concepts
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Extending Context Window of Large Language Models via Positional Interpolation
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
SPECTER: Document-level Representation Learning using Citation-informed Transformers
Tackling Distribution Shifts in Task-Oriented Communication with Information Bottleneck
Byte Pair Encoding is Suboptimal for Language Model Pretraining
Reviving The Classics: Active Reward Modeling in Large Language Model Alignment
MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Universal Language Model Fine-tuning for Text Classification
Improving Generalization Performance by Switching from Adam to SGD
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Rethinking Model Ensemble in Transfer-based Adversarial Attacks
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.