Reproduce “Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zeping Yu, Sophia Ananiadou
This paper introduces Comparative Neuron Analysis (CNA) to dissect the internal arithmetic mechanisms of LLMs, identifying a four-stage logic chain within Llama-7B for processing numerical operations. The analysis reveals specific attention heads and FFN neurons involved and demonstrates how LoRA fine-tuning amplifies relevant neuron activations.
Key insights
- Discovered a four-stage internal logic chain for arithmetic operations in LLMs: Feature Enhancing, Feature Transferring, Feature Predicting, and Prediction Enhancing.
- Identified that different arithmetic cases (memory, change-one, algorithmic) evoke specialized processing patterns within the model.
- Revealed that LoRA fine-tuning enhances prediction probabilities by amplifying coefficient scores of relevant FFN neurons, with shallow layer LoRA being more effective due to its early feature enhancement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.