Reproduce “Diverse Weight Averaging for Out-of-Distribution Generalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alexandre Ramé, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, Matthieu Cord
DiWA averages the weights of independently trained models to produce one network for out-of-distribution generalization, rather than averaging checkpoints from one run or predictions at inference. Its bias–variance–covariance–locality account links diversity to reduced covariance while requiring averageable weights. On DomainBed, LP-initialized DiWA† reaches 68.0% versus 66.9% for SWAD, but not every dataset and with unmatched initialization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.