Reproduce “Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wei Fang, James Glass
Researchers from MIT introduced TOOLQP, a framework that recasts tool retrieval for LLM agents as an iterative query planning process to navigate large, dynamic tool libraries. This approach achieves approximately 20% higher nDCG@10 compared to base retrievers and improves end-to-end tool calling success rates by 8-9% on complex benchmarks.
Key insights
- Recasting tool retrieval as an iterative query planning problem with interactive feedback allows LLM agents to bridge semantic gaps and uncover implicit inter-tool dependencies.
- Synthetic data generation, coupled with Reinforcement Learning with Verifiable Rewards (RLVR), enables effective training of a lightweight planning model without needing extensive real-world interaction data.
- Dynamic query planning, even with a small model, outperforms static, single-shot retrieval and expensive post-hoc reranking methods, especially for zero-shot generalization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.