Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
원문 초록
In offline multi-agent reinforcement learning (MARL), agents estimate policies from a given dataset. We study reward-poisoning attacks in this setting where an exogenous attacker modifies the rewards in the dataset before the agents see the dataset. The attacker wants to guide each agent into a nefarious target policy while minimizing the $L^p$ norm of the reward modification. Unlike attacks on single-agent RL, we show that the attacker can install the target policy as a Markov Perfect Dominant Strategy Equilibrium (MPDSE), which rational agents are guaranteed to follow. This attack can be significantly cheaper than separate single-agent attacks. We show that the attack works on various MARL agents including uncertainty-aware learners, and we exhibit linear programs to efficiently solve the attack problem. We also study the relationship between the structure of the datasets and the minimal attack cost. Our work paves the way for studying defense in offline MARL.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Robust Multi-Agent Reinforcement Learning with Model Uncertainty
Multi-Step Reinforcement Learning for Single Image Super-Resolution
Multirobot Coverage of Modular Environments
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
Using Fuzzy Logic to Learn Abstract Policies in Large-Scale Multi-Agent Reinforcement Learning
Task Independent Capsule-Based Agents for Deep Q-Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.