Reproduce “Hybrid Reasoning Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He, James Zou
HybridAgent lets a tool-using language-model agent reason in continuous hidden states and use text for tool calls and answers. HybridLink aligns hidden states with the model’s input-embedding space, while retained attention-cache entries carry context across turns. At 14B parameters, it scores 49.5 average on WebWalker versus ARPO’s 40.5, within evaluations of Qwen3-Base models on 11 benchmarks.
Key insights
- Latent reasoning can be integrated into multi-turn tool use without decoding every internal step into text; the paper argues that preserving the full KV cache maintains the hybrid prefix during continuation.
- Self-conditioned training improves performance beyond format training and adapter activation alone, addressing the gap between teacher-forced training and free-running latent inference.
- Replacing or reordering latent states reduces mathematical accuracy, suggesting that the content and sequence of those states matter, rather than the gains coming only from extra computation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.