Reproduce “$\mathrm{SO}(2)$-Equivariant Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Dian Wang, Robin Walters, Robert Platt

Researchers from Northeastern University introduced SO(2)-equivariant reinforcement learning algorithms that directly encode rotational symmetry into neural network architectures for robotic manipulation. The approach significantly improves sample efficiency and generalization, enabling solutions for complex tasks like block stacking and house building that proved intractable for conventional RL methods.

Key insights

  • Directly embedding SO(2) rotational symmetries into RL network architectures (equivariant networks) provides a more robust and effective mechanism for symmetry exploitation compared to learning symmetries through data augmentation.
  • For G-invariant MDPs, the inherent properties of optimal Q-functions being invariant and optimal policies being equivariant provide a principled theoretical basis for designing symmetric RL agents.
  • Achieving equivariance in both the actor and critic networks consistently yields the best performance, with the actor's equivariance contributing more significantly to overall performance gains.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.