A Targeted Acceleration and Compression Framework for Low bit Neural Networks
원문 초록
1 bit deep neural networks (DNNs), of which both the activations and weights are binarized , are attracting more and more attention due to their high computational efficiency and low memory requirement . However, the drawback of large accuracy dropping also restrict s its application. In this paper, we propose a novel Targeted Acceleration and Compression (TAC) framework to improve the performance of 1 bit deep neural networks W e consider that the acceleration and compression effects of binarizing fully connected layer s are not sufficient to compensate for the accuracy loss caused by it In the proposed framework, t he convolutional and fully connected layer are separated and optimized i ndividually . F or the convolutional layer s , both the activations and weights are binarized. For the fully connected layer s, the binarization operation is re placed by network pruning and low bit quantization. The proposed framework is implemented on the CIFAR 10, CIFAR 100 and ImageNet ( ILSVRC 12 ) datasets , and experimental results show that the proposed TAC can significantly improve the accuracy of 1 bit deep neural networks and outperforms the state of the art by more than 6 percentage points .
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models
Quantization-based Bounds on the Wasserstein Metric
Edge Computing for Physics-Driven AI in Computational MRI: A Feasibility Study
From Roots to Rewards: Dynamic Tree Reasoning with RL
DCR: Quantifying Data Contamination in LLMs Evaluation
U-RWKV: Lightweight medical image segmentation with direction-adaptive RWKV
토론
이 논문과 연결된 포럼 글이 아직 없습니다.