Reproduce “Revisiting In-context Learning Inference Circuit in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hakaze Cho, Mariko Kato, Yoshihiro Sakai, Naoya Inoue
A 3-step inference circuit, comprising input text encoding, semantics merging, and feature retrieval/copying, mechanistically explains In-context Learning (ICL) in real-world Large Language Models (LLMs). This framework from JAIST and RIKEN unifies understanding of phenomena like positional bias and noise robustness, while also characterizing auxiliary bypass mechanisms.
Key insights
- The proposed 3-step circuit provides a unified explanation for various ICL phenomena, including positional bias (from encoding), noise robustness (via 'label denoising'), and demonstration saturation (from submodular attention assignment).
- Transformer layers exhibit functional specialization during ICL, with early layers primarily handling input encoding and later layers focusing on semantics merging and feature retrieval for output preparation.
- Even when the primary 3-step circuit is completely ablated, 'bypass mechanisms' (e.g., parallel circuits, direct decoding through residual connections) contribute to ICL, preventing performance from dropping to zero-shot levels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.