Reproduce “Experience Rot: Experienced Agents Fail Where Novices Succeed” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhen Xu, Hang Yin, Zitao Zhang, Kevin Zhao, Xuefeng Liu
The authors study experience rot, when agents reuse a once-correct tool call after a silent interface change. They compare no experience, valid experience, and stale experience across six models and four changes in controlled services. With incomplete documentation, success was 63%, 100%, and 23%, respectively; these results concern short synthetic tasks, not arbitrary real-world updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.