Reproduce “Weights initialization of neural networks for function approximation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinwen Hu, Yunqing Huang, Nianyu Yi, Peimeng Yin
A reusable neural approximation framework is presented, pretraining neural networks on basis functions and integrating a domain mapping mechanism. This method achieves highly accurate function approximation with enhanced generalization across arbitrary domains and reduces training costs for diverse one- and two-dimensional functions.
Key insights
- Pretraining neural networks on fundamental basis functions provides a reusable and 'meaningful initialization prior' for subsequent approximation tasks, accelerating convergence.
- The proposed domain mapping mechanism is critical for enabling neural networks to generalize effectively and perform reliable extrapolation across arbitrary input ranges.
- Optimal architectural choices (e.g., single hidden layer with 1024 neurons) and the GELU activation function offer a balance of accuracy and computational efficiency for basis function approximation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.