Reproduce “On Measuring Localization of Shortcuts in Deep Networks” on OpenResearch

About this paper

Nikita Tsoy, Nikola Konstantinov

Researchers from INSAIT, Sofia University “St. Kliment Ohridski” developed a quantitative framework to measure the layer-wise localization of shortcut learning in deep networks. The study found that shallow layers primarily encode spurious features while deeper layers contribute to forgetting core features, and the developed metrics predict the efficacy of certain layer-wise mitigation strategies.

Key insights

• Shortcut learning is not concentrated in any single layer; rather, it is distributed throughout the network, with complex interactions between layers being crucial for its emergence.
• Shallow layers predominantly contribute to encoding spurious features, while deeper layers primarily contribute to forgetting core features.
• The localization of spurious feature encoding is strongly influenced by the dataset and model architecture, whereas core feature forgetting is more influenced by skew frequency and optimizer choice.

Summary by alphaXiv.