Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distillation for Online Ranking Systems
원문 초록
Knowledge Distillation (KD) is a powerful approach for compressing a large model into a smaller, more efficient model, particularly beneficial for latency-sensitive applications like recommender systems. However, current KD research predominantly focuses on Computer Vision (CV) and NLP tasks, overlooking unique data characteristics and challenges inherent to recommender systems. This paper addresses these overlooked challenges, specifically: (1) mitigating data distribution shifts between teacher and student models, (2) efficiently identifying optimal teacher configurations within time and budgetary constraints, and (3) enabling computationally efficient and rapid sharing of teacher labels to support multiple students. We present a robust KD system developed and rigorously evaluated on multiple large-scale personalized video recommendation systems within Google. Our live experiment results demonstrate significant improvements in student model performance while ensuring consistent and reliable generation of high quality teacher labels from a continuous data stream of data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Light distillation for Incremental Graph Convolution Collaborative Filtering
Revisiting Graph based Social Recommendation: A Distillation Enhanced Social Graph Network
A Reproducibility Study of Product-side Fairness in Bundle Recommendation
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition
NLGCL: Naturally Existing Neighbor Layers Graph Contrastive Learning for Recommendation
Boosting Parameter Efficiency in LLM-Based Recommendation through Sophisticated Pruning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.