Reproduce “All Models Are Miscalibrated, But Some Less So: Comparing Calibration with Conditional Mean Operators” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peter Moskvichev, Dino Sejdinovic
The authors introduce the conditional kernel calibration error (CKCE), a measure for comparing a classifier’s class probabilities with observed label distributions conditional on each probability vector. CKCE embeds these distributions in reproducing-kernel Hilbert spaces. In ImageNet validation comparisons, it ranks ViT lower than ResNet, unlike expected calibration error; these estimator rankings do not establish which model is truly better calibrated.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.