Reproduce “ReProbe: Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models” on OpenResearch

About this paper

Jingwei Ni, Ekaterina Fadeeva, Tianyi Wu, Mubashara Akhtar, Jiaheng Zhang, Elliott Ash, Markus Leippold, Timothy Baldwin and 3 more

ReProbe uses a compact classifier over a frozen language model’s hidden states, attention patterns, and logits to estimate the correctness of individual reasoning steps. With fewer than 10 million parameters, it often matches or exceeds much larger process reward models, improves best-of-N and beam-search reasoning, and delivers roughly 2.6–25× lower verification runtime in the reported experiments.

Key insights

• Internal representations contain useful signals about reasoning reliability: hidden-state probes consistently outperform inexpensive unsupervised uncertainty measures and are especially competitive on out-of-domain planning and question answering.
• Self-generated labels perform similarly to externally generated labels on average, suggesting that probe training can avoid much of the cost of human or large-model step annotation.
• ReProbe and process reward models capture complementary information; combining them or using ReProbe as a preliminary filter can reduce expensive PRM calls by about 31–56% while preserving comparable performance.

Summary by alphaXiv.