Papers with Code 아카이브

Knowledge Distillation

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

전체 기법 전체 데이터셋

하위 태스크: 2
평가표: 11
보존된 평가 기록: 90

Knowledge distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models) have higher knowledge capacity than small models, this capacity might not be fully utilized.

코드

관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.

https://github.com/hs-yn/daps

공식 코드

Dense 2D-3D Indoor Prediction with Sound via Aligned Cross-Modal Distillation

https://github.com/hsajjad/transformers

공식 코드

On the Effect of Dropping Layers of Pre-trained Transformer Models

https://github.com/hsd1503/RDPD

공식 코드

RDPD: Rich Data Helps Poor Data via Imitation

https://github.com/hse-lambda/digital-twin

공식 코드

Performance Modeling of Data Storage Systems using Generative Models

https://github.com/hsg-aiml/neurips_2021-weight_space_learning

공식 코드

Hyper-Representations for Pre-Training and Transfer Learning

https://github.com/hsg-aiml/neurips_2022-generative_hyper_representations

공식 코드

Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights

https://github.com/hsliu-initial/rok

공식 코드

Retrieval-Oriented Knowledge for Click-Through Rate Prediction

https://github.com/hsouri/battle-of-the-backbones

공식 코드

Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks

https://github.com/hsp-iit/prosthetic-grasping-simulation

공식 코드

Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis

https://github.com/hsw0929/hsd

공식 코드

Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks

https://github.com/htqin/bibench

공식 코드

BiBench: Benchmarking and Analyzing Network Binarization

https://github.com/htyao89/camera-incremental-object-reid

공식 코드

Camera-Incremental Object Re-Identification with Identity Knowledge Evolution

https://github.com/huangd1999/EffiBench

공식 코드

EffiBench: Benchmarking the Efficiency of Automatically Generated Code

https://github.com/huangowen/quantization-variation

공식 코드

Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision

https://github.com/huangowen/vvtq

공식 코드

Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision

https://github.com/huawei-noah/KD-NLP/tree/main/Minimax-kNN

공식 코드

Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax

https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TernaryBERT

공식 코드

TernaryBERT: Distillation-aware Ultra-low Bit BERT

https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT

공식 코드

TinyBERT: Distilling BERT for Natural Language Understanding

https://github.com/huawei-noah/SMARTS

공식 코드

CRITERIA: a New Benchmarking Paradigm for Evaluating Trajectory Prediction Models for Autonomous Driving

https://github.com/huawei-noah/hebo

공식 코드

AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisation

https://github.com/huawei-noah/kd-nlp

공식 코드

When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation

https://github.com/hubreb/imitkd_ast

공식 코드

Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts

https://github.com/hugcis/benchmark_learning_efficiency

공식 코드

Benchmarking Learning Efficiency in Deep Reservoir Computing

https://github.com/hugcis/incremental_tasks

공식 코드

Benchmarking Learning Efficiency in Deep Reservoir Computing

https://github.com/huggingface/distil-whisper

공식 코드

Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling

https://github.com/huggingface/swift-coreml-transformers

공식 코드

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

https://github.com/huggingface/transformers

공식 코드

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md

공식 코드

Pre-trained Summarization Distillation

https://github.com/hugozhl/hetu

공식 코드

Experimental Analysis of Large-scale Learnable Vector Storage Compression

https://github.com/hui-design/aand

공식 코드

Advancing Pre-trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detection

https://github.com/hulitaotom/Joint-Multi-Scale-Tone-Mapping-and-Denoising-for-HDR-Image-Enhancement

공식 코드

Joint Multi-Scale Tone Mapping and Denoising for HDR Image Enhancement

https://github.com/humza909/llm_survey

공식 코드

A Comprehensive Overview of Large Language Models

https://github.com/hunto/diffkd

공식 코드

Knowledge Diffusion for Distillation

https://github.com/hunto/dist_kd

공식 코드

Knowledge Distillation from A Stronger Teacher

https://github.com/hust-iof/efficient-dvs

공식 코드

Real-time Event Recognition of Long-distance Distributed Vibration Sensing with Knowledge Distillation and Hardware Acceleration

https://github.com/hust-linyi/LENAS

공식 코드

LENAS: Learning-based Neural Architecture Search and Ensemble for 3D Radiotherapy Dose Prediction

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.