Reproduce “Stopping Bayesian Optimization with Probabilistic Regret Bounds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
James T. Wilson
James T. Wilson introduces Probabilistic Regret Bounds (PRB), a novel model-based stopping rule for Bayesian Optimization that terminates the process when a solution is within a user-specified \u03b5 of the optimum with 1-\u03b4 probability. The method consistently achieves target success rates in experiments, often requiring fewer evaluations than baselines, but is sensitive to model mismatch.
Key insights
- It is feasible to replace arbitrary fixed-budget BO stopping with a user-centric criterion based on desired solution quality (\u03b5) and confidence (1-\u03b4).
- Robust statistical decision-making, even when relying on noisy Monte Carlo estimates, can provide strong theoretical guarantees for BO stopping criteria (convergence and correctness under the model).
- The reliability of model-based stopping rules is highly dependent on the accuracy and calibration of the underlying probabilistic model, with model mismatch being a primary challenge.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.