Reproduce “MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma and 3 more

MemAgent answers long-document questions by streaming text in chunks and rewriting a fixed-size, task-focused memory after each chunk. The final answer uses the question and accumulated memory. Reinforcement learning trains the model to retain answer-bearing evidence and discard distractors without changing its architecture. On synthetic RULER-HQA, RL-MemAgent-14B scores 71.09% at 3.5M tokens, versus 80.47% at 7K, demonstrating extrapolation on a related benchmark, not arbitrary documents.

Key insights

  • Modeling human-like selective memory, where only essential information from long documents is abstracted and retained, can enable LLMs to manage unbounded input lengths efficiently.
  • Reinforcement learning provides an effective end-to-end mechanism to train LLMs to learn optimal memory management policies for retention and compression, allowing for robust extrapolation.
  • By replacing the unbounded autoregressive history with a fixed-length memory, the computational cost for processing long texts can be reduced to linear complexity (O(N)), making it scalable.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.