Reproduce “Memory in Large Language Models: Mechanisms, Evaluation and Evolution” on OpenResearch

About this paper

Dianxing Zhang, Wendong Li, Kani Song, Jiaye Lu, Gang Li, Liuchun Yang, Sheng Li

Researchers at Digital China AI Research Institute developed a unified operational framework for understanding, evaluating, and governing Large Language Model (LLM) memory, establishing a four-way taxonomy and a three-setting parallel evaluation protocol. This work provides an auditable Dynamic Memory Management & Governance (DMM-Gov) framework and actionable engineering principles to enhance LLM deployment safety and compliance in high-stakes applications.

Key insights

• LLM "memory" is a multilayered collection of states, influenced by training, network structure, and system architecture, necessitating a holistic management approach.
• Expanding context windows alone is often insufficient for robust long-context capabilities; structured approaches like reordering and summary-replay are frequently more effective and cost-efficient.
• Retrieval recall is the primary performance bottleneck in Retrieval-Augmented Generation (RAG) systems, with improvements in reranking and grounding mechanisms synchronously boosting correctness and faithfulness.
• Current editing and unlearning techniques face limitations in achieving simultaneous effectiveness, locality, and scalability, pointing to the need for strategies like "small-step editing" and "external coverage first."

Summary by alphaXiv.