Reproduce “SciWorkBench: A Benchmark for Real Scientific Workflows” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuchen Zhang, Taolin Han, Jinghang Wang, Jinxin Wang, Yun Wu, Wai Yuet Chiu, Xiaogang Li, Yuting Fu and 14 more
SciWorkBench evaluates scientific agents on research-grounded tasks that test decisions across stages of experimental analysis without prescribing one path. Expert-validated workflows and rubrics assess whether outputs support conclusions from evidence. Across 98 tasks, GPT-5.6 Sol with Codex had the highest rubric-based mean score, 57.38; this is not a solved-task rate and applies only to tested setups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.