Reproduce “When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Suchit Gupte, Xueru Zhang, Mohammad Mahdi Khalili
Researchers from The Ohio State University investigated how weight pruning impacts Sparse Autoencoders (SAEs) used for Large Language Model (LLM) interpretability. They developed a theoretical framework based on "perturbation energy" to quantify pruning's effects, empirically demonstrating that activation-aware pruning methods preserve SAE fidelity and interpretability metrics significantly better than magnitude pruning across various LLM architectures.
Key insights
- Activation-aware pruning methods (WANDA, SPARSEGPT) are substantially more robust at preserving SAE interpretability and reconstruction fidelity than magnitude pruning, aligning with the developed perturbation energy theory.
- Middle layers of LLMs consistently exhibit elevated sensitivity to pruning across methods and sparsity levels, indicating a systematic structural vulnerability likely due to cumulative damage propagation.
- Standard language modeling benchmarks and aggregate SAE reconstruction metrics are insufficient to detect significant degradation in fine-grained interpretability properties, underscoring the need for specialized interpretability evaluation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.