Reproduce “Diffusion Tempering Improves Parameter Estimation with Probabilistic Integrators for Ordinary Differential Equations” on OpenResearch

About this paper

Jonas Beck, Nathanael Bosch, Michael Deistler, Kyra L. Kadhim, Jakob H. Macke, Philipp Hennig, Philipp Berens

Diffusion tempering uses a probabilistic ODE solver to improve parameter recovery by optimizing at a fixed diffusion level, then lowering diffusion in stages and reusing the estimate. In a simulated two-compartment, six-parameter Hodgkin–Huxley model, it correctly converged in 88% of 100 initializations, versus 0% for Runge–Kutta least squares, using pRMSE below 0.05; this tests known dynamics, not real recordings.

Key insights

• The diffusion hyperparameter (κ) in probabilistic numerical integrators can be leveraged as an explicit regularization knob, controlling the smoothness of the likelihood landscape and enabling a trade-off between broad exploration and precise optimization.
• A tempering schedule that gradually reduces κ allows gradient-based optimizers to navigate complex, rugged loss landscapes more effectively, overcoming local minima by leveraging the initial smoothing effect.
• By separating the optimization of parameters from the dynamic control of κ, the method successfully extends the robustness benefits of probabilistic integrators to more challenging and high-dimensional ODE models.

Summary by alphaXiv.