Reproduce “TreeHCA: Hindsight-Guided Exploration and Credit Assignment for Multi-Turn LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Debajoy Mukherjee, Jeremy Carleton, Sushil Vemuri, Srinivas Shakkottai, Dileep Kalathil
TreeHCA trains search agents using hindsight-guided branching and credit assignment. It revisits histories before turns that reduce reference-answer compatibility, samples alternatives, and weights outcomes to account for selection preferences. For Qwen3-8B multi-hop question answering, it reports 50.25% weighted exact match, versus 48.41% for AT²PO. The theoretical identities rely on specified sampling assumptions; the practical backup uses proxy scores on an adaptive tree.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.