Reproduce “Recurrent Harness for Agentic Continual Learning” on OpenResearch

About this paper

Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang

ReHarness revises an agent’s retrieved reflections after observing how they affect decisions in a task stream. It uses the reflection’s source trajectory to ground new content, while the later outcome supplies feedback; it changes textual memory, not model parameters. Across three benchmarks and three backbones, ReHarness reaches 0.240 average normalized gain, ranking first or tied in eight of nine combinations.

Key insights

• Treating stored experience as revisable—not merely accumulating or curating entries—can help correct advice exposed as ineffective by later use.
• Revisions were largely source-grounded in an automated analysis: over 95% of classified claims were supported by the original trajectory, with 0.8–2.1% leakage from the later trajectory.
• Performance depends on feedback quality: noisy rewards, as in the reported exploitable-poker case, can make it harder to distinguish poor advice from an unfavorable outcome.

Summary by alphaXiv.