Papers with Code 아카이브

Mixture-of-Experts

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

전체 기법전체 데이터셋

하위 태스크

0

평가표

0

보존된 평가 기록

0

코드

관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.

https://github.com/pwc-1/Paper-9/tree/main/2/nllb_moe

LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

https://github.com/pwc-1/Paper-9/tree/main/2/olmo

OLMoE: Open Mixture-of-Experts Language Models

https://github.com/pwc-1/Paper-9/tree/main/6/RHFL

Robust Federated Learning by Mixture of Experts

https://github.com/pwc-1/Paper-9/tree/main/jamba

Jamba: A Hybrid Transformer-Mamba Language Model

https://github.com/pwc-1/Paper-9/tree/main/jetmoe

JetMoE: Reaching Llama2 Performance with 0.1M Dollars

https://github.com/qwenlm/qwen1.5

Qwen2 Technical Report

https://github.com/ragerri/antidote-projections

No Language Left Behind: Scaling Human-Centered Machine Translation

https://github.com/ritwikgupta/tencent-hunyuan-large

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

https://github.com/robertjkeck2/EmoNet

Distilling the Knowledge in a Neural Network

https://github.com/robertjkeck2/EmoTe

Distilling the Knowledge in a Neural Network

https://github.com/robertjkeck2/pwdl

Distilling the Knowledge in a Neural Network

https://github.com/rohin-dasari/Tensorflow-Knowledge-Distillation

Distilling the Knowledge in a Neural Network

https://github.com/sail-sg/adan

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models

https://github.com/satyakees/FaultNet

Distilling the Knowledge in a Neural Network

https://github.com/scy-x/d3still

Distilling the Knowledge in a Neural Network

https://github.com/see--/speech_recognition

Distilling the Knowledge in a Neural Network

https://github.com/shadowpa0327/Palu

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

https://github.com/shawntan/SUT

Sparse Universal Transformer

https://github.com/shawntan/stickbreaking-attention

Scattered Mixture-of-Experts Implementation

https://github.com/shinome551/select_with_distillation

Distilling the Knowledge in a Neural Network

https://github.com/shriramsb/Distilling-the-Knowledge-in-a-Neural-Network

Distilling the Knowledge in a Neural Network

https://github.com/sjtu-ipads/powerinfer

Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters

https://github.com/stanford-futuredata/stk

MegaBlocks: Efficient Sparse Training with Mixture-of-Experts

https://github.com/stdereka/knowledge-distillation

Distilling the Knowledge in a Neural Network

https://github.com/sungnyun/LG-knowledge-distillation

Distilling the Knowledge in a Neural Network

https://github.com/t0nyliang/EEGMobile

Distilling the Knowledge in a Neural Network

https://github.com/tencent/hunyuan-large

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

https://github.com/tencent/tencent-hunyuan-large

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

https://github.com/terenceylchow124/Meme-MultiModal

Gated Multimodal Units for Information Fusion

https://github.com/thaonguyen19/modeldistillation-pytorch

Distilling the Knowledge in a Neural Network

https://github.com/trqminh/knowledge-distillation

Distilling the Knowledge in a Neural Network

https://github.com/unconst/MACH

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

https://github.com/usryokousha/mone-pytorch

Mixture of Nested Experts: Adaptive Processing of Visual Tokens

https://github.com/varungumma/fairseq

Distilling the Knowledge in a Neural Network

https://github.com/vicentvankor/sun-shine

Qwen2 Technical Report

https://github.com/vikparuchuri/marker

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.