Reproduce “IPR-1: Interactive Physical Reasoner” on OpenResearch

About this paper

Mingyu Zhang, Lifeng Zhuo, Tianxi Tan, Guocan Xie, Xian Nie, Yan Li, Renjie Zhao, Zizhu He and 3 more

Researchers at Shanghai Jiao Tong University introduced the Interactive Physical Reasoner (IPR), an agent that acquires human-like physical and causal reasoning from diverse interactive environments. IPR leverages a physics-centric action code (PhysCode) and integrates vision-language models with a world model, demonstrating robust zero-shot transfer across a new 1000+ game benchmark (G2U) and showing performance improvements that scale with increased interactive experience.

Key insights

• Generalizable physical and causal reasoning can be acquired through interaction with diverse environments, and this ability consistently improves with accumulated experience, indicating that agents can distill invariant mechanisms.
• A physics-centric latent action space (PhysCode) is essential for disentangling core dynamics from domain-specific visual appearances and action interfaces, enabling robust zero-shot transfer of physical understanding.
• Integrating world model predictions to reinforce VLM policies in an interactive loop effectively bridges the gap between high-level reasoning and low-level predictive grounding, leading to superior performance in complex physical tasks.

Summary by alphaXiv.