Reproduce “A Multi-Perspective Analysis of Memorization in Large Language Models” on OpenResearch

About this paper

Bowen Chen, Namgi Han, Yusuke Miyao

Memorization in large language models (LLMs) is explored from a micro-level perspective using the Pythia model suite, revealing non-linear scaling with a maximum capacity for memorization. The study identifies distinct "boundary effects" in input n-gram frequencies and output entropy that differentiate memorized from unmemorized content, and demonstrates the predictability of token-level memorization based on internal model states.

Key insights

• Memorization exhibits a non-linear increase with model size, suggesting a finite capacity, and most training data remains unmemorized even in large models.
• Distinct "boundary effects" in token n-gram frequencies (input dynamics) and decoding entropy (output dynamics) signal the transition between memorized and unmemorized content generation.
• LLMs form clusters for highly memorized content in embedding space, even for non-verbatim outputs, indicating the presence of "paraphrased memorization."

Summary by alphaXiv.