Reproduce “How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, Tanmoy Chakraborty
Researchers from IIT Delhi and IIT Bombay provided a mechanistic understanding of Chain-of-Thought (CoT) reasoning in Llama-2 7B, revealing that diverse CoT subtasks leverage overlapping neural components and that information processing undergoes a significant shift in later model layers. They found that multiple parallel pathways contribute to answer generation, drawing from different contextual sources.
Key insights
- LLMs employ overlapping sets of attention heads for diverse CoT subtasks (decision-making, copying, induction), with inductive reasoning heads being broadly effective.
- Information mixing for ontological relationships is a non-linear process that improves with model depth, peaking in mid-layers and not significantly benefiting from in-context examples in a zero-shot regime.
- A "functional rift" around the 16th decoder block in Llama-2 7B marks a transition from pretraining-prior driven processing in earlier layers to in-context-prior driven processing in later layers, where all answer-writing heads reside.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.