HSVI for zs-POSGs using Concavity, Convexity and Lipschitz Properties
원문 초록
Dynamic programming and heuristic search are at the core of state-of-the-art solvers for sequential decision-making problems. In partially observable or collaborative settings (\eg, POMDPs and Dec-POMDPs), this requires introducing an appropriate statistic that induces a fully observable problem as well as bounding (convex) approximators of the optimal value function. This approach has succeeded in some subclasses of 2-player zero-sum partially observable stochastic games (zs-POSGs) as well, but failed in the general case despite known concavity and convexity properties, which only led to heuristic algorithms with poor convergence guarantees. We overcome this issue, leveraging on these properties to derive bounding approximators and efficient update and selection operators, before deriving a prototypical solver inspired by HSVI that provably converges to an $\epsilon$-optimal solution in finite time, and which we empirically evaluate. This opens the door to a novel family of promising approaches complementing those relying on linear programming or iterative methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments
TextAtari: 100K Frames Game Playing with Language Agents
LLM-Stackelberg Games: Conjectural Reasoning Equilibria and Their Applications to Spearphishing
A Survey of Continual Reinforcement Learning
Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.