Reproduce “DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Aryan Mobiny, Hien V. Nguyen, Supratik Moulik, Naveen Garg, Carol C. Wu

This paper presents MC-DropConnect, a method that reinterprets DropConnect regularization as a computationally efficient approach for Bayesian uncertainty estimation in deep neural networks. The proposed method demonstrates improved prediction accuracy and higher quality uncertainty quantification compared to Monte Carlo Dropout across various classification and semantic segmentation tasks.

Key insights

  • DropConnect, typically used for regularization, can be reinterpreted as a computationally efficient variational approximation for Bayesian Neural Networks, enabling uncertainty quantification without adding model parameters.
  • The proposed MC-DropConnect method provides better-calibrated uncertainty estimates and marginally improved prediction accuracy compared to MC-Dropout, and achieves faster convergence during inference.
  • The introduction of novel metrics like Negative Predictive Value (NPV), True Positive Rate (TPR), and Uncertainty Accuracy (UA) based on AUC allows for objective and quantitative evaluation of uncertainty estimation techniques.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.