Reproduce “Joint Training of Deep Ensembles Fails Due to Learner Collusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alan Jeffares, Tennison Liu, Jonathan Crabbé, Mihaela van der Schaar
The authors compare independently trained neural networks combined at evaluation with networks trained jointly to optimize ensemble prediction. They explain that joint training can reward diversity produced by opposing learner shifts that cancel when averaged, rather than useful differences. On ImageNet, five ResNet-18 learners achieve 68.10% top-1 accuracy independently and 62.74% jointly; this comparison does not establish a universal rule.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.