Reproduce “Artifact-to-Capability: Diagnosing Breaks in Capability Formation of Self-Evolving Agents in a Controlled Environment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziqian Ban, Haoxiang Yang, Yanfei Zhang, Shixin Liang, Haoran Yu, Xu Lin, Shaobin Zhao, Xuran Li and 1 more
The authors test whether reusable artifacts form and extend task capabilities in three self-evolving Gomoku systems. They compare replayed moves, complete games, and version changes using a shared model and environment. Voyager and Reflexion fail the capability-formation checkpoint, while DGM passes at OB-3, but later versions do not extend it. These controlled results diagnose Gomoku implementations, not open-ended tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.