Reproduce “A Multiplicative Neural Network Architecture: Locality and Regularity of Approximation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hee-Sun Choi, Beom-Seok Han
A novel multiplicative neural network architecture (MMLP) is introduced, proving its universal approximation in Bessel potential spaces and demonstrating inherently more localized error structures and stable regularity-sensitive convergence compared to additive Multi-Layer Perceptrons.
Key insights
- A single multiplicative unit inherently generates a localized, mollifier-type kernel, as its support is formed by the intersection of geometrically confined strip-like activations, a fundamental distinction from additive MLP units.
- The MMLP architecture achieves universal approximation in Bessel potential spaces ($H_p^\gamma(R^m)$), indicating its capability to approximate functions with controlled regularity and integrability.
- Regularity-sensitive metrics, such as Zygmund-type seminorms, reveal that MMLPs provide more stable and sustained control over pointwise oscillations during training compared to standard MLPs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.