Reproduce “AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversification” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Geonwoo Cho, Jaemoon Lee, Jaegyun Im, Subi Lee, Jihwan Lee, Sundong Kim

Researchers from Gwangju Institute of Science and Technology and Seoul National University developed AMPED, an adaptive multi-objective projection method that balances exploration and skill diversification in unsupervised reinforcement learning. The system achieved superior performance on 12 downstream tasks from the URLB benchmark, surpassing state-of-the-art baselines by up to 35.01% in interquartile mean (IQM) scores.

Key insights

  • Explicitly resolving gradient conflicts between exploration and skill diversity objectives through gradient surgery leads to more effective and stable skill learning, addressing a fundamental challenge in multi-objective reinforcement learning.
  • Combining diverse intrinsic reward components (entropy, RND, AnInfoNCE) enables the acquisition of a skill repertoire that simultaneously achieves broad state space exploration and clear skill distinguishability.
  • Increased skill diversity can theoretically reduce the sample complexity required for fine-tuning by simplifying the task of identifying the most suitable skill for a given policy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.