Reproduce “Physically-Grounded Goal Imagination: Physics-Informed Variational Autoencoder for Self-Supervised Reinforcement Learning” on OpenResearch

About this paper

Lan Thi Ha Nguyen, Kien Ton Manh, Anh Do Duc, Nam Pham Hai

Researchers at FPT University developed Physics-Informed RIG (PI-RIG), a framework that integrates physical knowledge into self-supervised goal generation for reinforcement learning, leading to more efficient robotic skill acquisition. This method achieved up to a 74% improvement in task completion performance over prior methods like CC-RIG in visual robotic manipulation tasks.

Key insights

• Purely data-driven generative models in reinforcement learning often produce physically impossible goals, leading to wasted learning effort and inefficient skill acquisition.
• Integrating known physical laws and domain-specific constraints directly into the VAE architecture and goal sampling can effectively filter out implausible goals, improving goal quality.
• Decomposing the latent space into distinct components for intrinsic physics and environmental factors provides a more interpretable and controllable representation for generating achievable goals.

Summary by alphaXiv.