Reproduce “Identifiable Equivariant Networks are Layerwise Equivariant” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vahid Shahverdi, Giovanni Luca Marchetti, Georg Bökman, Kathlén Kohn
An identifiable deep neural network that functions equivariantly end-to-end must also exhibit layerwise equivariance, a finding that explains why such internal symmetries emerge in models trained on symmetric data. This work provides a general, architecture-agnostic theoretical framework for this relationship, applicable to various network types including MLPs and attention networks.
Key insights
- Parameter identifiability plays a critical role in mediating the relationship between end-to-end network equivariance and layerwise equivariance.
- The ubiquitous practice of designing neural networks in a layerwise equivariant manner is theoretically supported as a necessary condition for achieving overall equivariance in identifiable models.
- The observed emergence of layerwise equivariant representations in networks trained with symmetric data is a theoretically expected phenomenon, providing a mathematical explanation for this empirical finding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.