Reproduce “Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers” on OpenResearch

About this paper

Amit Ben-Artzy, Roy Schwartz

Research by Amit Ben-Artzy and Roy Schwartz from The Hebrew University of Jerusalem investigates the layer-wise importance of attention in decoder-based Large Language Models (LLMs), revealing a two-phase processing model. Their findings suggest lower layers primarily aggregate information from previous tokens, while upper layers consolidate this information, demonstrating robustness to manipulations of previous token representations in the top 30-50% of layers for many tasks.

Key insights

• LLMs exhibit a two-phase processing architecture: an initial "attend" phase in lower layers for aggregating information from previous tokens, followed by a "consolidate" phase in upper layers.
• Lower layers are highly sensitive to the integrity and content of previous token representations, while upper layers show increased robustness to manipulations of these representations.
• The attention mechanism's necessity for processing previous token information diminishes in higher layers for many tasks, but it remains crucial for tasks requiring precise, sequential reasoning like math.

Summary by alphaXiv.