Reproduce “LLMs cannot find reasoning errors, but can correct them given the error location” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gladys Tyen, Hassan Mansoor, Victor Cărbune, Peter Chen, Tony Mak
Large Language Models demonstrate a robust ability to correct reasoning errors when provided with the error's exact location, yet they struggle to reliably identify these errors themselves, achieving only about 52.87% accuracy in mistake finding. The research also introduces the BIG-Bench Mistake dataset to facilitate further study into detecting logical errors in LLM outputs.
Key insights
- LLMs' primary challenge in self-correction for reasoning tasks is reliably identifying the location of logical errors within their own Chain-of-Thought traces.
- When provided with precise error location information, LLMs demonstrate a robust capacity to correct reasoning mistakes and improve the accuracy of their outputs.
- Specialized, smaller classifiers, trained on out-of-domain data, can achieve higher mistake-finding accuracy than few-shot prompting of larger LLMs, indicating a viable path for error detection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.