Reproduce “Learning Concept Bottleneck Models from Mechanistic Explanations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Antonio De Santis, Schrasing Tong, Marco Brambilla, Lalana Kagal
Researchers developed a pipeline to construct Concept Bottleneck Models (CBMs) by extracting concepts from black-box model features using Sparse Autoencoders and then naming/annotating them with Multimodal Large Language Models. This approach leads to CBMs that consistently surpass existing baselines in classification accuracy and concept prediction quality across datasets, providing more faithful and concise explanations.
Key insights
- Concepts derived directly from a black-box model's learned representations are inherently more relevant, learnable, and predictive for the task than pre-defined external concepts.
- Integrating mechanistic interpretability techniques (SAEs) with MLLMs enables automated, scalable discovery and annotation of high-fidelity, interpretable concepts for CBMs.
- Controlling explanation sparsity through the NCC metric at a decision-level is crucial for ensuring faithful CBM explanations and effectively mitigating information leakage, leading to more transparent AI systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.