Reproduce “Fast Non-Episodic Adaptive Tuning of Robot Controllers with Online Policy Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
James A. Preiss, Fengze Xie, Yiheng Lin, Adam Wierman, Yisong Yue
M-GAPS tunes robot-controller gains online, updating them along a live trajectory rather than at episode boundaries. It recursively estimates sensitivities using derivatives from an approximate dynamics model, then uses them to update gains with O(1) computation per step. On a Crazyflie tracking a detuned figure-eight, it approached expert behavior after 2–3 laps (8–12 seconds), in this tested setting.
Key insights
- Non-episodic, model-based online policy optimization (M-GAPS) offers a superior balance of flexibility, data efficiency, and real-time adaptivity compared to both restrictive adaptive control and data-hungry RL.
- Continuous, single-trajectory adaptation (M-GAPS) avoids the pitfalls of episodic methods, such as sensitivity to episode length and large, discrete parameter updates that can induce instability.
- Utilizing approximate models with efficient analytic gradient computation (enabled by tools like SymForce) makes complex online optimization feasible for real-time execution on resource-constrained embedded systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.