Reproduce “Geometric Flow Models over Neural Network Weights” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ege Erdogan
This thesis develops geometric flow models to generate neural network weights, systematically addressing inherent symmetries to improve efficiency. It demonstrates these models can produce high-quality samples, enable effective Bayesian Model Averaging, and facilitate transfer learning across tasks and architectures.
Key insights
- Explicitly modeling neural network weight symmetries (both permutation and scaling) substantially enhances the effectiveness and efficiency of weight-space generative models.
- Defining velocity fields directly on the canonicalized product geometry via Riemannian Flow Matching yields performance advantages over simpler Euclidean approaches.
- These geometric flow models learn generalizable properties of optimization landscapes, enabling task-gradient guided transfer learning and architecture generalization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.