SGAD: Soft-Guided Adaptively-Dropped Neural Network
원문 초록
Deep neural networks (DNNs) have been proven to have many redundancies. Hence, many efforts have been made to compress DNNs. However, the existing model compression methods treat all the input samples equally while ignoring the fact that the difficulties of various input samples being correctly classified are different. To address this problem, DNNs with adaptive dropping mechanism are well explored in this work. To inform the DNNs how difficult the input samples can be classified, a guideline that contains the information of input samples is introduced to improve the performance. Based on the developed guideline and adaptive dropping mechanism, an innovative soft-guided adaptively-dropped (SGAD) neural network is proposed in this paper. Compared with the 32 layers residual neural networks, the presented SGAD can reduce the FLOPs by 77% with less than 1% drop in accuracy on CIFAR-10.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Simple is what you need for efficient and accurate medical image segmentation
Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs
FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation
Advances in Small-Footprint Keyword Spotting: A Comprehensive Review of Efficient Models and Algorithms
Structured Pruning and Quantization for Learned Image Compression
FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression
토론
이 논문과 연결된 포럼 글이 아직 없습니다.