Reproduce “Equivariant Goal Conditioned Contrastive Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Arsh Tangri, Nichols Crawford Taylor, Haojie Huang, Robert Platt

Researchers from Northeastern University developed Equivariant Contrastive RL (ECRL), a framework that integrates rotational equivariance into goal-conditioned reinforcement learning, leading to significant improvements in sample efficiency and spatial generalization for robotic manipulation tasks. ECRL consistently surpassed non-equivariant baselines across various simulated state-based and image-based environments, also proving effective in offline learning settings.

Key insights

  • Leveraging inherent geometric symmetries within goal-conditioned reinforcement learning significantly enhances sample efficiency and generalization.
  • Building a rotation-invariant critic is most effective by using aligned cyclic permutations of equivariant regular representation vectors, rather than global pooling, which preserves critical orientation information.
  • The theoretical property that optimal Q-functions are group-invariant and optimal policies are group-equivariant in Goal-Conditioned Group-Invariant MDPs provides a principled guide for architecture design.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.