Reproduce “Evaluating Uncertainty Calibration in Probabilistic Time Series Foundation Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Thomas Decker, Volker Tresp

The authors evaluate calibration in 14 zero-shot time-series foundation models across 55 short-term configurations, testing coverage overall, by forecast horizon, and among predictions with similar values. Horizon-wise error exceeded pooled error in 98% of model–dataset pairs and was 40% larger on average. Isotonic recalibration reduced errors but did not eliminate structured miscalibration; the study covers marginal, not joint, calibration.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.