Reproduce “Lower expected Fisher rank implies better PAC generalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marco Miani, Albert Kjøller Jacobsen, Albert Lopez i Serrano, Søren Hauberg, Georgios Arvanitidis
The authors derive a PAC-Bayes bound using a parameter distribution that diffuses along directions preserving predictions on observed data and adds Gaussian noise in directions that change them. Its complexity term includes expected Fisher rank, the number of nonzero eigenvalues of the Fisher information matrix, divided by sample size. With near-zero training loss, this term governs scaling; no experiments are reported.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.