Reproduce “A Decision-Theoretic Approach for Model Interpretability in Bayesian Framework” on OpenResearch

About this paper

Homayun Afrabandpey, Tomi Peltola, Juho Piironen, Aki Vehtari, Samuel Kaski

Researchers from Aalto University and Curious AI developed a decision-theoretic framework for Bayesian model interpretability, creating simpler surrogate models that approximate accurate, complex Bayesian reference models. This two-step approach achieves a more optimal balance between explanation fidelity and interpretability, generating more accurate and stable explanations that incorporate model uncertainty.

Key insights

• Interpretability is a user preference distinct from assumptions about the data generation process, necessitating a separation between the predictive model and its explanation.
• A decision-theoretic utility function can rigorously balance explanation fidelity and model interpretability, yielding a more optimal trade-off than embedding interpretability constraints directly into model priors.
• Leveraging an accurate, unconstrained reference model as the target for interpretation yields superior interpretable proxy models, as the proxy has a stronger and less biased signal to approximate.

Summary by alphaXiv.