Reproduce “Large Language Models as Common-Sense Heuristics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrey Borro, Patricia J Riddle, Michael W Barley, Michael J Witbrock
A planning method integrates large language models as common-sense heuristics within a local search algorithm, enabling an agent to plan for household tasks in a simulated 3D environment. This approach achieves an average task success rate of 69% in the VirtualHome environment, outperforming baselines like ProgPrompt by 22 percentage points, while consistently generating executable plans.
Key insights
- LLMs can effectively serve as intuitive common-sense heuristics within structured search algorithms, balancing semantic understanding with formal rigor.
- This "LLM-modulo" framework enables robust and executable plan generation, overcoming limitations of direct LLM-based planning for embodied tasks.
- LLMs struggle with specific formal world model constraints, action semantics, and logical sub-puzzles, even when understanding the overall task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.