Reproduce “Martingale Posterior Neural Networks for Fast Sequential Decision Making” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Gerardo Duran-Martin, Leandro Sánchez-Betancourt, Álvaro Cartea, Kevin Murphy

Researchers at the University of Oxford and Google DeepMind introduced Martingale Posterior Neural Networks (MPNNs), including the HiLoFi variant, for fast, online sequential decision-making by directly updating the posterior predictive distribution rather than parameter-space posteriors. This approach leads to inference speeds 10-100 times faster than classical Thompson sampling while maintaining or improving decision performance across various tasks.

Key insights

  • Decoupling Bayesian decision-making from explicit parameter-space posterior inference by focusing on the one-step-ahead predictive distribution significantly reduces computational complexity.
  • Combining extended Kalman filter principles with structured, low-rank covariance approximations (e.g., HiLoFi's full-rank last layer and low-rank hidden layers) enables scalable, numerically stable, and efficient online updates for neural network parameters.
  • The random-walk dynamics for parameters prevent uncertainty collapse and facilitate adaptation in non-stationary environments, while the framework provides principled uncertainty quantification directly in the outcome space.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.