Reproduce “Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aditya Sharma, Ananya Gupta, Chengyu Wang, Chiamaka Adebayo, Jakub Kowalski
Researchers at IIT Bombay, IISc, San Francisco State University, University of Lagos, and University of Warsaw developed the Causal World Model Induction (CWMI) framework to imbue Large Language Models with a causal understanding of physical dynamics. This framework integrates a trainable Causal Physics Module with a frozen LLM and uses an intervention-based loss, achieving 89.4% zero-shot accuracy on PIQA and a 94.1% accuracy with an 87.6% Causal Consistency Score on the new PhysiCa-Bench.
Key insights
- Explicitly inducing a specialized, differentiable causal physics model into an LLM's architecture is more effective for robust physical reasoning than relying solely on fine-tuning general-purpose LLMs.
- An intervention-based training objective (L_causal) is critical for moving LLMs beyond statistical correlation to learn genuine cause-and-effect relationships and counterfactual reasoning.
- Architecturally separating linguistic understanding (frozen LLM) from physical simulation (trainable CPM) provides modularity that enhances performance and interpretability for complex physical reasoning tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.