Reproduce “The Representation Theory of Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marco Antonio Armenta, Pierre-Marc Jodoin
Researchers from Université de Sherbrooke developed a novel mathematical framework for neural networks based on quiver representation theory, providing a universal, exact description for diverse architectures and their data processing. This framework re-derives properties like positive scale invariance and defines a moduli space whose complex dimension (#E° - #˜V) offers a theoretical measure for network capacity and generalization.
Key insights
- Neural network computations are fundamentally algebraic and combinatorial, evidenced by network isomorphisms preserving the network function, implying an infinite number of distinct networks can compute the same mapping.
- Data processing within a neural network can be understood as mapping input data into distinct isomorphism classes within the network's specific moduli space.
- The network function can be decomposed into a map from the input space to the moduli space, followed by a map from the moduli space to the network's output, formalizing a modified manifold hypothesis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.