Reproduce “MemPO: Self-Memory Policy Optimization for Long-Horizon Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruoran Li, Xinghua Zhang, Haiyang Yu, Shitong Duan, Xiang Li, Wenxin Xiang, Chonghua Liao, Xudong Guo and 2 more
Researchers from Tsinghua University and Alibaba Group introduced MemPO, a self-memory policy optimization framework for LLM agents that enables autonomous memory summarization within an RL setting. This approach reduces token consumption by up to 73.12% and improves performance on multi-objective tasks with an F1 score gain of 25.98 percentage points over a non-memory baseline.
Key insights
- Treating memory management as an intrinsic, explicit action enables proactive, task-aligned information retention, moving beyond passive external retrieval.
- A memory-specific reward based on the effective information content (conditional probability of correct answer given memory) provides crucial, finer-grained feedback for optimizing memory summarization.
- Jointly optimizing memory summarization with reasoning and tool-use actions within a unified reinforcement learning framework leads to more efficient and coherent agent behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.