Reproduce “From Evidence to Action: How Tool-Using Agents Fail” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hongzhan Lin, Shidong Cao, Ziyang Luo, Wenhao Chai, Mong-Li Lee, Wynne Hsu
SafeActBench evaluates whether agents gather entity- and state-specific evidence before acting and carry actual results into later steps. Its deterministic evaluator scores trajectories across 656 synthetic cases and five protocols. On the same 100 V1 cases, GLM-5.2 with ZCode scored 99% static accuracy but 33% interactive exact case success, a 66-point gap that does not isolate its cause.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.