Reproduce “SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis” on OpenResearch

About this paper

Yuqing Yang, Alexander Schmatz, Zhaozhao Ma, Changkyu Choi, Robert Jenssen, Shujian Yu

Researchers developed SMILE, a framework for self-explainable multimodal medical diagnosis, which intrinsically generates modality-specific, feature-level explanations while achieving high predictive performance. The framework demonstrated strong diagnostic accuracy and identified clinically relevant features across various medical imaging, omics, and clinical datasets.

Key insights

• Self-explainability can be intrinsically integrated into multimodal deep learning by formulating the problem within the Information Bottleneck principle, allowing for simultaneous optimization of prediction and explanation generation.
• Tailoring explainers to operate directly on the input space of each modality, combined with differentiable top-k selection, enables the generation of granular, instance-wise, and modality-specific feature explanations.
• The Information Bottleneck's compression term, in addition to fostering compact representations, theoretically contributes to better generalization bounds, suggesting that self-explainability does not necessarily introduce an accuracy penalty.

Summary by alphaXiv.