Reproduce “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps” on OpenResearch

About this paper

Karen Simonyan, Andrea Vedaldi, Andrew Zisserman

The authors visualize supervised image classifiers by optimizing inputs for class prototypes and differentiating class scores with respect to pixels for image-specific saliency maps. They combine saliency with color-based GraphCut segmentation to produce object masks and bounding boxes. The weakly supervised pipeline achieved 46.4% top-5 localization error on ILSVRC-2013, versus 29.9% for the challenge winner; this is a bounding-box localization result, not a segmentation-accuracy score.

Key insights

• Gradient-based optimization can effectively generate coherent visual representations of what a supervised ConvNet "thinks" a particular class looks like.
• Image-specific class saliency maps, derived from gradients, provide a computationally efficient way to identify and localize discriminative regions in an image for a given classification.
• The proposed gradient-based visualization methods are formally connected to and generalize Deconvolutional Networks, offering a unifying theoretical perspective on understanding ConvNet activations.

Summary by alphaXiv.