Reproduce “Caption-Driven Explainability: Probing CNNs for Bias via CLIP” on OpenResearch

About this paper

Patrick Koller, Amil V. Dravid, Guido M. Schuster, Aggelos K. Katsaggelos

Researchers from Northwestern University, UC Berkeley, and Eastern Switzerland University of Applied Sciences developed a caption-based explainable AI method that integrates a standalone convolutional neural network into CLIP via a "network surgery" technique. This approach effectively identifies the dominant conceptual features a model uses for prediction, thereby detecting potential biases and enhancing robustness against covariate shift.

Key insights

• Concept-level explanations, facilitated by natural language captions, offer a deeper and more accurate understanding of model reasoning than pixel-based saliency maps, especially when detecting reliance on spurious features.
• The method enables proactive detection of model biases and vulnerabilities to covariate shift before deployment, allowing for timely debiasing and improved model reliability.
• Identifying the dominant concepts a CNN uses is a critical prerequisite for deploying robust and trustworthy machine vision models, particularly in high-stakes applications like medical imaging.

Summary by alphaXiv.