Reproduce “Learning When Human Decisions Matter: Executor-Relative Consultation Value for LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuli Jiang, Khushbu Pahwa, Yuting Zhang, Yi Zhang, Wei Xia, Stefano Soatto
CVD allocates scarce human consultations by predicting which inputs improve executor outcomes. It learns values from offline executor rollouts, scoring sets jointly in batch tasks. On batch Animals at budget 3, normalized gain was 0.58 versus 0.25 under Sonnet 5, significantly favoring CVD over Local Value. Local Value remained competitive in other settings, and on sequential Animals its numerical lead was not significant.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.