Fixed-point optimization of deep neural networks with adaptive step size retraining
원문 초록
Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performance even with 2- or 3-bit precision when quantized weights are fine-tuned by retraining. We propose an improved fixedpoint optimization algorithm that estimates the quantization step size dynamically during the retraining. In addition, a gradual quantization scheme is also tested, which sequentially applies fixed-point optimizations from high- to low-precision. The experiments are conducted for feed-forward deep neural networks (FFDNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs).
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Compress Any Segment Anything Model (SAM)
OpenDPDv2: A Unified Learning and Optimization Framework for Neural Network Digital Predistortion
EdgeCodec: Onboard Lightweight High Fidelity Neural Compressor with Residual Vector Quantization
any4: Learned 4-bit Numeric Representation for LLMs
CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.