Reproduce “A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Octi Zhang, Mateo Guaman Castro, Patrick Yin, Ignacio Dagnino, Abhishek Gupta, Rosario Scalise, Byron Boots
Success Guided Sampling (SGS) allocates training episodes across a fixed set of task configurations according to recent success rates, emphasizing situations with intermediate success while retaining nonzero sampling probability for all configurations. In settings, it improves results: Anymal-D locomotion reaches 73% success at 1 million environments, versus 54% with PLR. Hardware nut-and-bolt transfer succeeds in 18 of 48 trials.
Key insights
- A policy’s observed success rate can serve as a simple signal for allocating practice: intermediate-difficulty configurations are emphasized without imposing a fixed task order.
- Adaptive sampling depends on the available reset distribution; for rod-in-hole insertion, removing near-goal resets prevented learning, while removing reaching resets had little effect in the tested setup.
- Sampling improvements are most apparent on challenging tasks; on simpler tasks at sufficiently large scales, uniform sampling and other methods can also reach high success.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.