Debiasing Samples from Online Learning Using Bootstrap
원문 초록
It has been recently shown in the literature that the sample averages from online learning experiments are biased when used to estimate the mean reward. To correct the bias, off-policy evaluation methods, including importance sampling and doubly robust estimators, typically calculate the conditional propensity score, which is ill-defined for non-randomized policies such as UCB. This paper provides a procedure to debias the samples using bootstrap, which doesn't require the knowledge of the reward distribution and can be applied to any adaptive policies. Numerical experiments demonstrate the effective bias reduction for samples generated by popular multi-armed bandit algorithms such as Explore-Then-Commit (ETC), UCB, Thompson sampling (TS) and $\epsilon$-greedy (EG). We analyze and provide theoretical justifications for the procedure under the ETC algorithm, including the asymptotic convergence of the bias decay rate in the real and bootstrap worlds.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Minimum Empirical Divergence for Sub-Gaussian Linear Bandits
RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions
Cramming Contextual Bandits for On-policy Statistical Evaluation
Representative Action Selection for Large Action-Space Meta-Bandits
Steering Generative Models with Experimental Data for Protein Fitness Optimization
Addressing Missing Data Issue for Diffusion-based Recommendation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.