Reproduce “Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering” on OpenResearch

About this paper

Qingru Zhang, Xiaodong Yu, Chandan Singh, Xiaodong Liu, Liyuan Liu, Jianfeng Gao, Tuo Zhao, Dan Roth and 1 more

AutoPASTA (Automatic Post-hoc Attention Steering Approach) enhances Large Language Model (LLM) contextual faithfulness and performance in knowledge-intensive tasks like open-book Question Answering. It automatically identifies key contextual information and then explicitly steers the LLM's attention to it during inference, yielding consistent and substantial performance improvements across various LLMs and QA tasks without requiring retraining or human annotation.

Key insights

• Explicitly steering an LLM's attention to automatically identified key sentences offers more robust performance gains than merely appending them to the input prompt.
• The identified effective attention heads exhibit strong out-of-domain generalization, suggesting they possess a general utility for contextual comprehension across different tasks.
• Both the automatic identification of key contextual information and its explicit highlighting through attention manipulation are critical and complementary for improving LLM faithfulness.

Summary by alphaXiv.