arXiv:2411.12758
공식 코드 · Papers with Code
An exploration of the effect of quantisation on energy consumption and inference time of StarCoder2
원문 초록
This study examines quantisation and pruning strategies to reduce energy consumption in code Large Language Models (LLMs) inference. Using StarCoder2, we observe increased energy demands with quantization due to lower throughput and some accuracy losses. Conversely, pruning reduces energy usage but impairs performance. The results highlight challenges and trade-offs in LLM model compression. We suggest future work on hardware-optimized quantization to enhance efficiency with minimal loss in accuracy.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation
Structured Pruning and Quantization for Learned Image Compression
Efficient Speech Translation through Model Compression and Knowledge Distillation
DipSVD: Dual-importance Protected SVD for Efficient LLM Compression
RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models
EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.