Reproduce “Optimal Neural Network Approximation for High-Dimensional Continuous Functions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ayan Maiti, Michelle Michelle, Haizhao Yang

A neural network architecture is developed that approximates any high-dimensional continuous function to arbitrary accuracy using an Elementary Universal Activation Function, reducing the parameter count to an optimal O(d) from O(d^2). This construction uses 10889d + 10887 nonzero parameters, a theoretical efficiency supported by a proven lower bound on network width.

Key insights

  • Employing a particular variant of the Kolmogorov Superposition Theorem enables a functional decomposition that inherently leads to a lower parameter count for approximation.
  • The Elementary Universal Activation Function's inherent ability to approximate any univariate continuous function to arbitrary precision with a fixed, small network is leveraged for all network components.
  • The overall network's compositional structure, building complex approximations from simpler, reusable EUAF-based subnetworks, is key to its parameter efficiency.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.