Reproduce “Agents' Last Exam” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yiyou Sun, Xinyang Han, Weichen Zhang, Yuanbo Pang, Tianyu Wang, Yuhan Cao, Yixiao Huang, Xiang Liu and 371 more

Agents’ Last Exam (ALE) evaluates whether AI agents can complete computer-mediated professional workflows and produce verifiable deliverables. Experts’ completed projects become runnable tasks in software environments, where agents’ output artifacts are scored against hidden references or rubrics. Codex with GPT-5.5 achieves a 23.3% full-pass rate on ALE-CLI, compared with 82% on Terminal-Bench; this cross-benchmark contrast measures benchmark performance, not job readiness.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.