Reproduce “Confidence Calibration for Convolutional Neural Networks Using Structured Dropout” on OpenResearch

About this paper

Zhilu Zhang, Adrian V. Dalca, Mert R. Sabuncu

Structured dropout improves uncertainty estimates in convolutional neural networks by removing correlated spatial regions, channels, or residual layers during stochastic prediction. On SVHN, CIFAR-10, and CIFAR-100, these variants usually achieved lower expected calibration error than ordinary MC dropout and deterministic models, while also improving uncertainty-based active learning on CIFAR-10.

Key insights

• Calibration benefits from diversity among stochastic predictors, not merely from adding dropout noise; the error–ambiguity decomposition provides the paper’s theoretical motivation.
• Structured dropout encourages sampled CNNs to rely on different representations, producing more disagreement and complementary errors than standard unit-wise dropout.
• There is a trade-off between diversity and the accuracy of each sampled model, so the best dropout structure and rate depend on the dataset and architecture.

Summary by alphaXiv.