Reproduce “LLM-First Search: Self-Guided Exploration of the Solution Space” on OpenResearch

About this paper

Nathan Herr, Tim Rocktäschel, Roberta Raileanu

Researchers from UCL's Centre for AI develop LLM-First Search (LFS), a novel approach that empowers Large Language Models to autonomously control their own search process by making internal decisions about exploration versus exploitation rather than relying on fixed hyperparameters, achieving superior performance on Countdown and Sudoku reasoning tasks with 47.37% win rate on difficult Countdown problems (versus 32.63% for MCTS) and demonstrating the only successful solutions on complex 6x6 Sudoku puzzles, while requiring fewer computational tokens and exhibiting better scalability when paired with stronger models like o3-mini where LFS achieves 78.95% win rate compared to MCTS's 41.05%, effectively eliminating the need for manual hyperparameter tuning that plagues existing search-augmented LLM methods like Monte Carlo Tree Search and Best-First Search.

Key insights

• LLMs can effectively self-regulate their search strategy when given proper prompting frameworks for exploration decisions and action evaluation
• Self-guided exploration scales better with stronger models, with LFS benefiting more from model improvements than traditional search methods
• Internal LLM control over search generates smaller, more focused search trees compared to methods like MCTS that tend toward excessive exploration

Summary by alphaXiv.