Reproduce “Preparing for the Unknown: Learning a Universal Policy with Online System Identification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenhao Yu, Jie Tan, C. Karen Liu, Greg Turk
Researchers at Georgia Tech and Google Brain developed UP-OSI, a method for training robotic control policies entirely in simulation that adapt online to unknown dynamic models. This system consistently performs comparably to or better than policies given true parameters, even demonstrating an ability to generalize to parameters outside its training range.
Key insights
- Explicitly parameterizing the control policy with dynamic model parameters (μ) allows it to specialize and adapt to a wide range of system dynamics.
- Training the Online System Identification (OSI) component iteratively with 'bad cases' (where the policy operates based on erroneous parameter predictions) is crucial for its robustness and ability to recover from initial prediction errors.
- The method relies solely on simulated data for training, making it highly sample-efficient and eliminating the need for costly and risky real-world data collection during learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.