Detection Augmented Bandit Procedures for Piecewise Stationary MABs: A Modular Approach
원문 초록
Conventional Multi-Armed Bandit (MAB) algorithms are designed for stationary environments, where the reward distributions associated with the arms do not change with time. In many applications, however, the environment is more accurately modeled as being nonstationary. In this work, piecewise stationary MAB (PS-MAB) environments are investigated, in which the reward distributions associated with a subset of the arms change at some change-points and remain stationary between change-points. Our focus is on the asymptotic analysis of PS-MABs, for which practical algorithms based on change detection (CD) have been previously proposed. Our goal is to modularize the design and analysis of such CD-based Bandit (CDB) procedures. To this end, we identify the requirements for stationary bandit algorithms and change detectors in a CDB procedure that are needed for the modularization. We assume that the rewards are sub-Gaussian. Under this assumption and a condition on the separation of the change-points, we show that the analysis of CDB procedures can indeed be modularized, so that regret bounds can be obtained in a unified manner for various combinations of change detectors and bandit algorithms. Through this analysis, we develop new modular CDB procedures that are order-optimal. We compare the performance of our modular CDB procedures with various other methods in simulations.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Precision Spatio-Temporal Feature Fusion for Robust Remote Sensing Change Detection
Be the Change You Want to See: Revisiting Remote Sensing Change Detection Practices
HydroChronos: Forecasting Decades of Surface Water Change
Temporal-Spectral-Spatial Unified Remote Sensing Dense Prediction
Non-Registration Change Detection: A Novel Change Detection Task and Benchmark Dataset
Building-Guided Pseudo-Label Learning for Cross-Modal Building Damage Mapping
토론
이 논문과 연결된 포럼 글이 아직 없습니다.