Reproduce “Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Junwoo Chang, Minwoo Park, Joohwan Seo, Roberto Horowitz, Jongmin Lee, Jongeun Choi

Researchers at Yonsei University and UC Berkeley developed Partially Equivariant Reinforcement Learning (PE-RL) to enable robust and sample-efficient learning in environments where symmetries are only partially observed. Their methods, PE-DQN and PE-SAC, achieved improved sample efficiency and maintained stability, outperforming strictly equivariant and approximate equivariant baselines across diverse discrete and continuous control tasks, especially in settings with significant symmetry-breaking.

Key insights

  • Local violations of group symmetry in MDPs induce global value errors that propagate through repeated Bellman backups.
  • The PI-MDP framework provides a theoretical mechanism to mitigate error propagation by interpolating between the true MDP and its group-invariant approximation.
  • Learning to dynamically gate between equivariant and non-equivariant representations, based on local predictions of symmetry-breaking, is crucial for robust and efficient learning in partially symmetric environments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.