Toward Risk-based Optimistic Exploration for Cooperative Multi-Agent Reinforcement Learning
원문 초록
The multi-agent setting is intricate and unpredictable since the behaviors of multiple agents influence one another. To address this environmental uncertainty, distributional reinforcement learning algorithms that incorporate uncertainty via distributional output have been integrated with multi-agent reinforcement learning (MARL) methods, achieving state-of-the-art performance. However, distributional MARL algorithms still rely on the traditional $\epsilon$-greedy, which does not take cooperative strategy into account. In this paper, we present a risk-based exploration that leads to collaboratively optimistic behavior by shifting the sampling region of distribution. Initially, we take expectations from the upper quantiles of state-action values for exploration, which are optimistic actions, and gradually shift the sampling region of quantiles to the full distribution for exploitation. By ensuring that each agent is exposed to the same level of risk, we can force them to take cooperatively optimistic actions. Our method shows remarkable performance in multi-agent settings requiring cooperative exploration based on quantile regression appropriately controlling the level of risk.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
2048: Reinforcement Learning in a Delayed Reward Environment
Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study
Decentralizing Multi-Agent Reinforcement Learning with Temporal Causal Information
Robust Multi-Agent Reinforcement Learning with Model Uncertainty
Multi-Step Reinforcement Learning for Single Image Super-Resolution
Multirobot Coverage of Modular Environments
토론
이 논문과 연결된 포럼 글이 아직 없습니다.