Optimal channel selection with discrete QCQP
원문 초록
Reducing the high computational cost of large convolutional neural networks is crucial when deploying the networks to resource-constrained environments. We first show the greedy approach of recent channel pruning methods ignores the inherent quadratic coupling between channels in the neighboring layers and cannot safely remove inactive weights during the pruning procedure. Furthermore, due to these inactive weights, the greedy methods cannot guarantee to satisfy the given resource constraints and deviate with the true objective. In this regard, we propose a novel channel selection method that optimally selects channels via discrete QCQP, which provably prevents any inactive weights and guarantees to meet the resource constraints tightly in terms of FLOPs, memory usage, and network size. We also propose a quadratic model that accurately estimates the actual inference time of the pruned network, which allows us to adopt inference time as a resource constraint option. Furthermore, we generalize our method to extend the selection granularity beyond channels and handle non-sequential connections. Our experiments on CIFAR-10 and ImageNet show our proposed pruning method outperforms other fixed-importance channel pruning methods on various network architectures.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SepPrune: Structured Pruning for Efficient Deep Speech Separation
Differentiable Channel Selection in Self-Attention For Person Re-Identification
SlimSeiz: Efficient Channel-Adaptive Seizure Prediction Using a Mamba-Enhanced Network
Learning Binary Color Filter Arrays with Trainable Hard Thresholding
Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification
Quantifying Spatial Domain Explanations in BCI using Earth Mover's Distance
토론
이 논문과 연결된 포럼 글이 아직 없습니다.