Reproduce “Accelerate Bench: Evaluating Agents for Application-Level Program Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bowen Liu, Yuhan Li, Nuo Chen, Weile Luo, Miao Peng, Qintong Zhang, Qifan Zhang, Jia Li and 2 more
Accelerate Bench evaluates whether agents can speed scientific-computing workflows while preserving outputs. Tasks specify workloads and correctness constraints but leave optimization choices to agents, who can revise code and settings. On version 1.1, Claude Sonnet 5.5 scored 67.4% in one run. A QiboTN replay found deferred loading helped only with broad dispatch in the tested program and circuits.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.