Reproduce “World Modelling Improves Language Model Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shangmin Guo, Omar Darwiche Domingues, Raphaël Avalos, Aaron Courville, Florian Strub
A methodology combining Dynamics Modelling (DyMo) and Self-Verification Sampling (SVS) allows Large Language Models to internally predict action outcomes in stateful tool-use environments, lessening dependence on external verification. This led to 93.62% state prediction accuracy, an 83.62% unweighted overall success rate, and enabled reliable refusal with approximately 94.5% precision.
Key insights
- LLMs can effectively learn and internalize environment dynamics through dedicated training on state prediction data, enabling them to anticipate action outcomes and reduce reliance on costly external verification.
- Integrating dynamics modelling into both SFT and RL significantly mitigates hallucination and improves policy reliability, leading to more robust tool-use agents.
- An internal environment model, combined with self-verification sampling, enables scalable planning during LLM inference and a critical 'refusal' capability, enhancing overall system safety and trustworthiness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.