Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learning
원문 초록
Thompson sampling (TS) has emerged as a robust technique for contextual bandit problems. However, TS requires posterior inference and optimization for action generation, prohibiting its use in many online platforms where latency and ease of deployment are of concern. We operationalize TS by proposing a novel imitation-learning-based algorithm that distills a TS policy into an explicit policy representation, allowing fast decision-making and easy deployment in mobile and server-based environments. Using batched data collected under the imitation policy, our algorithm iteratively performs offline updates to the TS policy, and learns a new explicit policy representation to imitate it. Empirically, our imitation policy achieves performance comparable to batch TS while allowing more than an order of magnitude reduction in decision-time latency. Buoyed by low latency and simplicity of implementation, our algorithm has been successfully deployed in multiple video upload systems for Meta. Using a randomized controlled trial, we show our algorithm resulted in significant improvements in video quality and watch time.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
OWMM-Agent: Open World Mobile Manipulation With Multi-modal Agentic Data Synthesis
Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity
Time-Unified Diffusion Policy with Action Discrimination for Robotic Manipulation
Stable Thompson Sampling: Valid Inference via Variance Inflation
Graph-Structured Data Analysis of Component Failure in Autonomous Cargo Ships Based on Feature Fusion
토론
이 논문과 연결된 포럼 글이 아직 없습니다.