Reproduce “On the disintegration of the stochastic majority vote: From PAC-Bayesian bounds to a self-bounding algorithm” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Julien Bastian, Benjamin Leblanc, Pascal Germain, Amaury Habrard, Guillaume Metzler, Emilie Morvant, Paul Viallard
A disintegrated PAC-Bayes framework turns stochastic majority-vote guarantees into certificates for a single deterministic ensemble sampled from a distribution over voter weights. Across binary and multiclass benchmarks, the resulting methods produce much tighter bounds than classical surrogate analyses, with predictive accuracy close to stochastic alternatives and reduced training cost for the Rényi-based variant.
Key insights
- Applying PAC-Bayes at the level of ensemble parameters, rather than individual voters, preserves direct control of majority-vote risk while removing prediction-time randomization.
- DIS-R generally provides tighter deterministic certificates, whereas DIS-V offers more stable and faster optimization through a distribution-level divergence penalty.
- Derandomization incurs only a modest certificate gap relative to stochastic majority-vote methods and does not materially reduce test accuracy in the reported experiments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.