Reproduce “Language Models Can Explain Visual Features via Steering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Javier Ferrando, Enrique Lopez-Cuena, Pablo Agustin Martin-Torres, Daniel Hinjos, Anna Arias-Duart, Dario Garcia-Gasulla
Barcelona Supercomputing Center researchers introduced "Steering," a causal intervention method to generate natural language explanations for Sparse Autoencoder (SAE) features in vision models by directly manipulating the model's internal state. A hybrid approach, "Steering-informed Top-k," achieved the highest explanation quality across metrics by combining this causal signal with contextual input, enhancing scalability and reducing contextual biases compared to input-centric methods.
Key insights
- Causal intervention via "Steering" can effectively isolate and elicit visual concepts represented by individual SAE features, providing a more direct and less biased form of explanation compared to correlation-based methods.
- Combining causal intervention (Steering) with contextual input (Top-k images) in "Steering-informed Top-k" yields superior, more robust, and semantically aligned feature explanations, leveraging synergistic benefits.
- The quality and informativeness of automatically generated explanations for vision features improve with the increasing scale and capabilities of the underlying large language model used as the explainer.
- Direct feature intervention significantly mitigates contextual biases prevalent in input-based explanations, allowing for more faithful interpretation of lower-level or background features often overshadowed by prominent foreground objects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.