Reproduce “Temporal Consistency for LLM Reasoning Process Error Identification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiacheng Guo, Yue Wu, Jiahao Qiu, Kaixuan Huang, Xinzhe Juan, Ling Yang, Mengdi Wang
Researchers from Princeton University and University of Michigan introduce a temporal consistency method for identifying reasoning errors in Large Language Models that enables small distilled models (7B/8B parameters) to outperform 70B parameter models and GPT-4 on mathematical process verification tasks through iterative self-checking and multi-agent verification, achieving improvements of up to 46.6% on benchmark datasets without requiring additional training.
Key insights
- Temporal consistency through iterative refinement provides a new form of test-time scaling complementary to parallel sampling
- Stricter stability requirements in the convergence criteria correlate with improved verification accuracy
- The combination of multiple independent verifiers and temporal self-checking is particularly effective for harder problems
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.