Reproduce “Structured and Balanced Multi-Component and Multi-Layer Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shijun Zhang, Hongkai Zhao, Yimin Zhong, Haomin Zhou
Researchers from Florida Atlantic University, Duke University, Auburn University, and Georgia Institute of Technology developed Multi-component and Multi-layer Neural Networks (MMNNs) to efficiently approximate complex functions, especially those with high-frequency components and localized features. The architecture, which trains only a subset of parameters, demonstrates superior accuracy and faster convergence compared to traditional deep networks across various challenging tasks, including solving PDEs.
Key insights
- By fixing random feature weights and training only output coefficients per layer, MMNNs achieve a balanced trade-off between expressive power and optimization efficiency.
- The multi-layer composition allows for a hierarchical decomposition of functions, enabling the network to progressively learn and combine approximations of simpler components.
- ResMMNNs (MMNNs with residual connections) extend the approach to very deep networks, mitigating training issues and further enhancing approximation capabilities for highly complex functions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.