Convex Markov Games: A New Frontier for Multi-Agent Reinforcement Learning
원문 초록
Behavioral diversity, expert imitation, fairness, safety goals and others give rise to preferences in sequential decision making domains that do not decompose additively across time. We introduce the class of convex Markov games that allow general convex preferences over occupancy measures. Despite infinite time horizon and strictly higher generality than Markov games, pure strategy Nash equilibria exist. Furthermore, equilibria can be approximated empirically by performing gradient descent on an upper bound of exploitability. Our experiments reveal novel solutions to classic repeated normal-form games, find fair solutions in a repeated asymmetric coordination game, and prioritize safe long-term behavior in a robot warehouse environment. In the prisoner's dilemma, our algorithm leverages transient imitation to find a policy profile that deviates from observed human play only slightly, yet achieves higher per-player utility while also being three orders of magnitude less exploitable.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Guiding LLM Decision-Making with Fairness Reward Models
UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments
LLM-Stackelberg Games: Conjectural Reasoning Equilibria and Their Applications to Spearphishing
LumiCRS: Asymmetric Contrastive Prototype Learning for Long-Tail Conversational Movie Recommendation
A Survey of Continual Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.