MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure
원문 초록
Multi-agent reinforcement learning (MARL) algorithms often struggle to find strategies close to Pareto optimal Nash Equilibrium, owing largely to the lack of efficient exploration. The problem is exacerbated in sparse-reward settings, caused by the larger variance exhibited in policy learning. This paper introduces MESA, a novel meta-exploration method for cooperative multi-agent learning. It learns to explore by first identifying the agents' high-rewarding joint state-action subspace from training tasks and then learning a set of diverse exploration policies to "cover" the subspace. These trained exploration policies can be integrated with any off-policy MARL algorithm for test-time tasks. We first showcase MESA's advantage in a multi-step matrix game. Furthermore, experiments show that with learned exploration policies, MESA achieves significantly better performance in sparse-reward tasks in several multi-agent particle environments and multi-agent MuJoCo environments, and exhibits the ability to generalize to more challenging tasks at test time.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
PMAT: Optimizing Action Generation Order in Multi-Agent Reinforcement Learning
Maximum Entropy Reinforcement Learning with Diffusion Policy
Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning
Offline Multi-agent Reinforcement Learning via Score Decomposition
CAE: Repurposing the Critic as an Explorer in Deep Reinforcement Learning
SrSv: Integrating Sequential Rollouts with Sequential Value Estimation for Multi-agent Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.