Papers with Code 아카이브
Network Pruning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 0
- 평가표
- 5
- 보존된 평가 기록
- 34
Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.
Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks
기법
관련 논문에서 소개한 기법입니다. 이 태스크에서의 평가 여부는 평가 결과에서 확인하세요.
SqueezeNet
SqueezeNet is a convolutional neural network that employs design strategies to reduce the number of parameters, notably with the use of fire modules that "squeeze" parameters using 1x1 convolutions.
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
SLR
Please enter a description about the method here
Enabling Retrain-free Deep Neural Network Pruning using Surrogate Lagrangian Relaxation
Movement Pruning
Movement Pruning is a simple, deterministic first-order weight pruning method that is more adaptive to pretrained model fine-tuning. Magnitude pruning can be seen as utilizing zeroth-order information (absolute value) of the running model. In contrast, movement pruning methods are where importance is derived from first-order information. Intuitively, instead of selecting weights that are far from zero, we retain connections that are moving away from zero during the training process.
Targeted Dropout
Please enter a description about the method here
HRank
HRank is a filter pruning method that explores the High Rank of the feature map in each layer (HRank). The proposed HRank is inspired by the discovery that the average rank of multiple feature maps generated by a single filter is always the same, regardless of the number of image batches CNNs receive. Based on HRank, the authors develop a method that is mathematically formulated to prune filters with low-rank feature maps.
LLM-Rank
LLM-Rank: A Graph Theoretical Approach to Pruning Large Language Models
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.