Reproduce “Promote, Suppress, Iterate: How Language Models Answer One-to-Many Factual Queries” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianyi Lorena Yan, Robin Jia
USC researchers elucidated the "promote-then-suppress" mechanism in Llama-3-8B-Instruct and Mistral-7B-Instruct-v0.2 when answering one-to-many factual queries, demonstrating how attention and MLP layers coordinate to promote all potential answers while suppressing previously generated ones. This process was consistently observed across multiple datasets and prompt templates.
Key insights
- LMs utilize a consistent "promote-then-suppress" mechanism for 1MKR: initially promoting all possible answers, then actively suppressing those already generated to prevent repetition.
- Attention layers primarily propagate subject information in middle layers and actively suppress previously generated answers in later layers, while MLP layers consistently promote all relevant answers and amplify suppression signals.
- Knowledge recall and repetition avoidance are interwoven processes, with individual attention heads contributing to both promotion and suppression rather than being performed by distinct, specialized components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.