Reproduce “Principled Human-AI Collaboration in Agentic Workflows” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Young Hyun Cho, Will Wei Sun
The authors use pilot-answer expert judgments to select verifier thresholds and attempt limits that meet an error tolerance while maximizing task release. On MultiHiertt, at a 5% tolerance, automation averaged 2.50% with 1,000 evaluations and 24.17% with 5,000, while released-answer error stayed near 1.20%. These held-out results average 30 calibrations and apply to this workflow and dataset.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.