Reproduce “Small Data, Big Decisions: Model Selection in the Small-Data Regime” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jorg Bornschein, Francesco Visin, Simon Osindero
The authors test whether small-data trials can rank neural-network architectures for larger-data use by training sufficiently large candidates on subsets from the same distribution and comparing held-out performance. Across tested datasets and models, relative rankings remain broadly stable beyond the interpolation threshold, given well-tuned optimization. On ImageNet, 128,000-example ResNets used 4–7× fewer gradient steps than full-data training under its stopping schedule.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.