Reproduce “CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adir Dayan, Yam Eitan, Haggai Maron
CrossGMN learns to refine a smaller network’s initialization using both a source network and the target initialization. It represents their parameters as a joint graph with symmetry-preserving cross-edges, then predicts an update in one feedforward pass. For MNIST SIREN compression from [2, 32, 32, 1] to [2, 16, 1], it achieves 8.89× median distillation-step speedup, not a universal or elapsed-time speedup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.