Off-Policy Exploitability-Evaluation in Two-Player Zero-Sum Markov Games
원문 초록
Off-policy evaluation (OPE) is the problem of evaluating new policies using historical data obtained from a different policy. In the recent OPE context, most studies have focused on single-player cases, and not on multi-player cases. In this study, we propose OPE estimators constructed by the doubly robust and double reinforcement learning estimators in two-player zero-sum Markov games. The proposed estimators project exploitability that is often used as a metric for determining how close a policy profile (i.e., a tuple of policies) is to a Nash equilibrium in two-player zero-sum games. We prove the exploitability estimation error bounds for the proposed estimators. We then propose the methods to find the best candidate policy profile by selecting the policy profile that minimizes the estimated exploitability from a given policy profile class. We prove the regret bounds of the policy profiles selected by our methods. Finally, we demonstrate the effectiveness and performance of the proposed estimators through experiments.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DOLCE: Decomposing Off-Policy Evaluation/Learning into Lagged and Current Effects
Trajectory World Models for Heterogeneous Environments
Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning
Minimum Empirical Divergence for Sub-Gaussian Linear Bandits
Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation
Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferences
토론
이 논문과 연결된 포럼 글이 아직 없습니다.