Reproduce “Positional Fragility in LLMs: How Offset Effects Reshape Our Understanding of Memorization Risks” on OpenResearch

About this paper

Yixuan Xu, Antoni-Joan Solergibert i Llaquet, Antoine Bosselut, Imanol Schlag

Researchers at ETH AI Center and EPFL uncovered "positional fragility" in large language models, demonstrating that verbatim memorization is highly dependent on a prefix's position within the context window, with short prefixes at the very beginning being most effective. They show that shifting sensitive training data deeper into context effectively suppresses extractable memorization without compromising model utility, simultaneously revealing a link between memorization failures and text degeneration.

Key insights

• LLM memorization exhibits "positional fragility," where verbatim recall is strongest for short prefixes at the context window's beginning (offset 0) and sharply declines as prefixes are shifted deeper into the sequence.
• Failures to retrieve memorized content, whether due to high positional offset or low exposure frequency, directly contribute to text degeneration characterized by repetitive and incoherent outputs.
• Training with smaller batch sizes leads to reduced verbatim recall and improved language coherence, suggesting a beneficial implicit regularization effect on generalization and memorization.

Summary by alphaXiv.