Reproduce “EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yufei He, Juncheng Liu, Yue Liu, Yibo Li, Tri Cao, Zhiyuan Hu, Xinxing Xu, Bryan Hooi

A new framework, EvoTest, enables AI agents to continuously learn and improve during deployment by holistically evolving their entire configuration, including policy prompts, structured memory, hyperparameters, and tool-use routines. Evaluated on the newly introduced Jericho Test-Time Learning (J-TTL) benchmark, EvoTest consistently surpassed both gradient-free and gradient-based adaptation methods in complex interactive environments.

Key insights

  • Test-time learning is demonstrably effective for improving agent performance on complex tasks, with learning-based methods consistently outperforming static agents.
  • Holistic, whole-system evolution across multiple agent components (prompt, memory, tools, hyperparameters) effectively addresses complex performance bottlenecks that single-channel adaptation methods cannot resolve.
  • Leveraging rich, narrative episode transcripts for semantic credit assignment offers a more data-efficient and robust learning mechanism than relying on sparse scalar rewards for gradient-based methods, particularly in sparse-reward environments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.