Reproduce “PIMbot: Policy and Incentive Manipulation for Multi-Robot Reinforcement Learning in Social Dilemmas” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shahab Nikkhoo, Zexin Li, Aritra Samanta, Yufei Li, Cong Liu
PIMbot is a framework that systematically investigates the manipulation of multi-robot reinforcement learning systems in social dilemmas by defining and implementing two forms of policy and incentive manipulation. The research shows these manipulations can either enhance cooperative learning, achieving 25-37.5% faster convergence in specific scenarios, or lead to collective task failure.
Key insights
- Manipulation in multi-robot systems can have a dual impact, sometimes unexpectedly accelerating collective learning even when driven by self-interest, as seen with partial communication.
- Incentive-driven MARL systems are vulnerable to various forms of manipulation targeting communication channels or individual agent policies, which can severely degrade collective performance.
- Environmental redundancy in task requirements emerges as a crucial factor in system resilience, allowing multi-robot teams to withstand certain policy manipulations by individual adversarial agents.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.