Reproduce “Retrieval-Driven Memory Reconsolidation for Long-Term LLM Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuanyi Song, Yukai Wang, Xinbei Ma, Zhihui Fu, Jianghao Lin, Weiwen Liu, Jun Wang, Huarong Deng and 2 more

REALM helps agents reuse information by storing typed units in a graph and revising links after answers using evidence and feedback. It changes graph topology, not memory text. On LoCoMo, REALM scores 75.97% versus MAGMA’s 68.80%; removing reconsolidation lowers scores by 2.01 points on LoCoMo and 2.13 on LongMemEval, showing an internal effect rather than sole cause of gains.

Key insights

  • Retrieval-driven reconsolidation improves organization and use of already retrieved evidence more than it increases the amount of evidence discovered: answer accuracy after evidence retrieval rose by 20.83% on LoCoMo and 19.23% on LongMemEval.
  • Removing reconsolidation reduced accuracy by 2.01 points on LoCoMo and 2.13 points on LongMemEval, with the largest benefits appearing in multi-hop, open-domain, preference, and knowledge-update questions.
  • Causal and logical edges were overrepresented in retrieved evidence, while associative links mainly provided broad connectivity; most LoCoMo evidence was found at the seed layer, unlike the deeper searches often needed on LongMemEval.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.