Reproduce “Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tong Zheng, Chengsong Huang, Runpeng Dai, Yun He, Rui Liu, Xin Ni, Huiwen Bao, Kaishen Wang and 4 more
A method called Parallel-Probe introduces 2D probing as a diagnostic interface to monitor collective dynamics across parallel LLM reasoning branches, dynamically controlling inference width and depth. This approach reduces sequential token costs by over 30% and total token costs by over 20% while preserving or improving reasoning accuracy across various models and benchmarks.
Key insights
- 2D probing revealed that LLM reasoning accuracy is non-monotonically dependent on the balance between width and depth, rather than solely on the total token budget.
- Parallel reasoning branches exhibit highly heterogeneous, long-tailed generation lengths, with a few 'outlier' branches disproportionately contributing to computational cost.
- Global consensus, derived from majority voting across parallel branches, stabilizes significantly earlier than the completion of the longest individual reasoning paths (average convergence onset ratio of 0.31).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.