Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines
원문 초록
Deep learning as a means to inferencing has proliferated thanks to its versatility and ability to approach or exceed human-level accuracy. These computational models have seemingly insatiable appetites for computational resources not only while training, but also when deployed at scales ranging from data centers all the way down to embedded devices. As such, increasing consideration is being made to maximize the computational efficiency given limited hardware and energy resources and, as a result, inferencing with reduced precision has emerged as a viable alternative to the IEEE 754 Standard for Floating-Point Arithmetic. We propose a quantization scheme that allows inferencing to be carried out using arithmetic that is fundamentally more efficient when compared to even half-precision floating-point. Our quantization procedure is significant in that we determine our quantization scheme parameters by calibrating against its reference floating-point model using a single inference batch rather than (re)training and achieve end-to-end post quantization accuracies comparable to the reference model.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models
Edge Computing for Physics-Driven AI in Computational MRI: A Feasibility Study
From Roots to Rewards: Dynamic Tree Reasoning with RL
DCR: Quantifying Data Contamination in LLMs Evaluation
U-RWKV: Lightweight medical image segmentation with direction-adaptive RWKV
MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.