PCAS: Pruning Channels with Attention Statistics for Deep Network Compression
원문 초록
Compression techniques for deep neural networks are important for implementing them on small embedded devices. In particular, channel-pruning is a useful technique for realizing compact networks. However, many conventional methods require manual setting of compression ratios in each layer. It is difficult to analyze the relationships between all layers, especially for deeper models. To address these issues, we propose a simple channel-pruning technique based on attention statistics that enables to evaluate the importance of channels. We improved the method by means of a criterion for automatic channel selection, using a single compression ratio for the entire model in place of per-layer model analysis. The proposed approach achieved superior performance over conventional methods with respect to accuracy and the computational costs for various models and datasets. We provide analysis results for behavior of the proposed criterion on different datasets to demonstrate its favorable properties for channel pruning.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SepPrune: Structured Pruning for Efficient Deep Speech Separation
Differentiable Channel Selection in Self-Attention For Person Re-Identification
SlimSeiz: Efficient Channel-Adaptive Seizure Prediction Using a Mamba-Enhanced Network
Learning Binary Color Filter Arrays with Trainable Hard Thresholding
Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification
Quantifying Spatial Domain Explanations in BCI using Earth Mover's Distance
토론
이 논문과 연결된 포럼 글이 아직 없습니다.