Reproduce “$σ$Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni, José Miguel Hernández-Lobato, Kamil Ciosek

σTransfer transfers Laplace prior-precision selection from a smaller neural network to a wider target by combining μP with width-dependent prior-covariance scaling. This stabilizes the prior kernel across widths, supporting more stable posterior uncertainty under stated conditions. On MNIST, a width-128 proxy searched about 5,000 times faster than a width-4096 target, with 0.002 target test-NLL degradation; one target posterior remains necessary.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.