Reproduce “Mind's Eye: Grounded Language Model Reasoning through Simulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te-Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, Andrew M. Dai
Google Research and Dartmouth College developed Mind's Eye, a paradigm that integrates external physics simulations into language models' inference processes to enhance their physical reasoning capabilities. This approach yielded a 46.0% absolute accuracy improvement in few-shot settings on average and enabled smaller LMs to achieve performance comparable to much larger ungrounded models.
Key insights
- Providing direct, ground-truth physical experience via simulation is more effective for robust physical reasoning than relying solely on internal or text-derived knowledge within LMs.
- Decoupling general reasoning from domain-specific physical knowledge allows LMs to leverage their language capabilities more effectively when supplied with accurate external evidence.
- The simulation-augmented approach significantly improves the efficiency of LMs, enabling smaller models to achieve performance comparable to or even surpassing much larger, ungrounded LMs by offloading complex physics modeling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.