Reproduce “Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nemanja Rakicevic, Antoine Cully, Petar Kormushev
Policy Manifold Search (PoMS) introduces a neuroevolutionary method that leverages a learned low-dimensional manifold in policy network parameter space to improve diversity-based policy search. This approach consistently achieved higher behavior coverage on complex continuous control tasks, demonstrating a relative increase in median coverage of 15% for Bipedal-Walker and 9% for Bipedal-Kicker over state-of-the-art baselines.
Key insights
- Searching within a learned low-dimensional manifold in policy network parameter space is more efficient for diversity-based neuroevolution, leading to higher behavior coverage.
- Non-linear manifold learning, as provided by AutoEncoders, is more effective than linear methods like PCA for capturing complex intrinsic structures relevant to policy behaviors.
- Using the decoder's Jacobian for heteroscedastic scaling of latent space mutations is critical for ensuring controlled and effective exploration, preventing erratic policy changes upon reconstruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.