Reproduce “Learning the Right Abstraction: Neural Reduced Dynamics for Complex Robot Control” on OpenResearch

About this paper

Harry Zhang, Dan Negrut

Researchers from the University of Wisconsin–Madison introduced a Neural Reduced Dynamics (NRD) framework, designed to enable efficient reinforcement learning for complex robots by creating fast, task-specific learned dynamics models from high-fidelity physics simulations. This approach achieved speedups of up to 58,400 times over direct simulation, allowing policies to be trained in minutes and demonstrating successful closed-loop transfer to complex robotic tasks like HMMWV trajectory tracking across diverse terrains and precise arm manipulation.

Key insights

• The 'right abstraction' for robot control is task-dependent and must focus on preserving control-relevant physics rather than compressing the full simulator state.
• Explicitly partitioning information within the dynamics model (learning only uncertain dynamics, analytically recovering deterministic relationships) is crucial for efficiency and effectiveness.
• Conditioning learned dynamics on an explicit context signal (e.g., terrain type) allows a single 'generalist' policy to robustly control complex robots across diverse and even previously unseen operating regimes.
• Closed-loop policy transfer back to the high-fidelity simulator serves as the definitive test for validating the utility and accuracy of the learned reduced dynamics model.

Summary by alphaXiv.