Reproduce “ART: Automatic multi-step reasoning and tool-use for large language models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, Marco Tulio Ribeiro
ART enables frozen large language models to automatically generate multi-step reasoning as interpretable programs, seamlessly integrating external tools to enhance performance on complex tasks without requiring fine-tuning.
Key insights
- Structured program-based reasoning inherently improves LLM performance over free-form Chain-of-Thought methods, even without explicit tool use.
- Dynamically integrating external tools is crucial for LLMs to overcome limitations in arithmetic precision, factual accuracy, and complex logical deduction, contributing significantly to overall task performance.
- The interpretable program format facilitates efficient human feedback, enabling rapid and substantial performance improvements through minor manual edits without costly LLM retraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.