Reproduce “Singularity-Bench: Can Agents Build Agents?” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yang Zhou, Krish Agarwal, Zhuoming Chen, Zerui Wang, Zhenyu Zhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng and 11 more

Singularity-Bench tests agents’ ability to build systems across six phases of AI development. Agents receive minimal code and fixed work budgets; their systems are compared with mature references on the same hardware. Among 11 agents, Opus 5.5 with Claude Code is the only one reported to match or beat references on all six tasks, with a Singularity-Index of 1.251. The benchmark is a proxy for recursive self-improvement, not a full test.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.