Reproduce “Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents” on OpenResearch

About this paper

Xian Yeow Lee, Sambit Ghadai, Kai Liang Tan, Chinmay Hegde, Soumik Sarkar

Iowa State University and NYU researchers developed Spatiotemporally Constrained Action Space Attacks, which introduce non-myopic (Look-ahead) adversarial strategies that perturb Deep Reinforcement Learning (DRL) agent actions over time. These attacks effectively degrade DRL agent performance in continuous control tasks more severely than myopic methods and identify specific, vulnerable action dimensions for improved diagnostic insights.

Key insights

• Non-myopic (LAS) action space attacks, which account for future dynamics, are consistently more effective at degrading DRL agent performance than myopic (MAS) attacks for a given adversarial budget.
• The attack methodology serves as a diagnostic tool, identifying specific action dimensions (e.g., Lunar Lander's 'Up-Down' thruster) that are disproportionately targeted and critical for agent performance.
• The choice of temporal projection norm (e.g., L2 vs. L1) impacts attack strategy: L2 leads to more continuous, sustained perturbations that prevent recovery, while L1 produces sparser, intense spikes allowing for potential agent stabilization.

Summary by alphaXiv.