Post-Training 4-bit Quantization on Embedding Tables
원문 초록
Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entities increases, the embedding components can easily contain millions of parameters and become the bottleneck in both storage and inference due to large memory consumption. This work focuses on post-training 4-bit quantization on the continuous embeddings. We propose row-wise uniform quantization with greedy search and codebook-based quantization that consistently outperforms state-of-the-art quantization approaches on reducing accuracy degradation. We deploy our uniform quantization technique on a production model in Facebook and demonstrate that it can reduce the model size to only 13.89% of the single-precision version while the model quality stays neutral.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
RecGPT: A Foundation Model for Sequential Recommendation
A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation
Deep Learning Model Acceleration and Optimization Strategies for Real-Time Recommendation Systems
A Reproducibility Study of Product-side Fairness in Bundle Recommendation
MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Compress Any Segment Anything Model (SAM)
토론
이 논문과 연결된 포럼 글이 아직 없습니다.