Reproduce “Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nikolaus Salvatore, Hao Wang, Qiong Zhang

Large language models develop the "lost-in-the-middle" effect as an emergent property from adapting to specific information retrieval demands during training, similar to human memory biases. Researchers at Rutgers University showed that training on mixed memory tasks yields U-shaped recall curves, driven by autoregressive architectures and attention sinks.

Key insights

  • The "lost-in-the-middle" effect is an emergent property arising from LLMs' adaptation to varying information retrieval demands during training, mirroring human memory's primacy and recency effects.
  • Primacy effects are strongly linked to the autoregressive nature of models (e.g., causal masking), as evidenced by their presence in decoder-only and RNN models but absence in bidirectional T5 architectures.
  • Attention sinks are not merely artifacts but play a functional role, selectively supporting long-term memory retrieval and the primacy effect, with less impact on short-term recall.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.