Reproduce “$σ$Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni, José Miguel Hernández-Lobato, Kamil Ciosek
σTransfer transfers Laplace prior-precision selection from a smaller neural network to a wider target by combining μP with width-dependent prior-covariance scaling. This stabilizes the prior kernel across widths, supporting more stable posterior uncertainty under stated conditions. On MNIST, a width-128 proxy searched about 5,000 times faster than a width-4096 target, with 0.002 target test-NLL degradation; one target posterior remains necessary.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.