Reproduce “Learning to Explain: An Information-Theoretic Perspective on Model Interpretation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianbo Chen, Le Song, Martin J. Wainwright, Michael I. Jordan
The L2X framework introduces an information-theoretic method for instance-wise feature selection to interpret black-box machine learning models. It leverages mutual information maximization and a learnable explainer network to identify critical features for individual predictions, demonstrating superior performance on complex datasets and achieving real-time explanation efficiency compared to prior techniques.
Key insights
- Defining an "informative" feature subset through mutual information provides a principled, theoretically grounded objective for model interpretation.
- Learning a global explainer network allows for highly efficient, real-time instance-wise explanations by amortizing training costs across numerous interpretations.
- Combining variational approximations with continuous relaxation techniques (Gumbel-softmax) makes it possible to optimize complex, discrete feature selection objectives using standard gradient-based methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.