Reproduce “Quasi-Equivariant Metanetworks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Viet-Hoang Tran, An Nguyen, Benoît Guérand, Thieu N. Vo, Tan M. Nguyen
Researchers from the National University of Singapore and FPT Software AI Center developed a quasi-equivariant framework for metanetworks, enabling them to better capture functional identity of neural networks by allowing outputs to be functionally equivalent rather than strictly identical under group actions. This approach consistently improved performance, such as achieving a Kendall's au of 0.926 for CNN generalization prediction, with only a 1-5% increase in parameters compared to strictly equivariant baselines.
Key insights
- Strict equivariance is a sufficient but not necessary condition for metanetworks to preserve functional identity; quasi-equivariance offers a more flexible path to this goal while boosting expressivity.
- Modulating the output of an equivariant metanetwork with a learnable, data-dependent group element allows for capturing more nuanced functional equivalences.
- Statistical features of neural network weights and biases provide an effective, compact input for learning the dynamic group elements required for quasi-equivariance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.