On Periodic Functions as Regularizers for Quantization of Neural Networks
원문 초록
Deep learning models have been successfully used in computer vision and many other fields. We propose an unorthodox algorithm for performing quantization of the model parameters. In contrast with popular quantization schemes based on thresholds, we use a novel technique based on periodic functions, such as continuous trigonometric sine or cosine as well as non-continuous hat functions. We apply these functions component-wise and add the sum over the model parameters as a regularizer to the model loss during training. The frequency and amplitude hyper-parameters of these functions can be adjusted during training. The regularization pushes the weights into discrete points that can be encoded as integers. We show that using this technique the resulting quantized models exhibit the same accuracy as the original ones on CIFAR-10 and ImageNet datasets.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Compress Any Segment Anything Model (SAM)
OpenDPDv2: A Unified Learning and Optimization Framework for Neural Network Digital Predistortion
EdgeCodec: Onboard Lightweight High Fidelity Neural Compressor with Residual Vector Quantization
any4: Learned 4-bit Numeric Representation for LLMs
CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.