Reproduce “On the Dataless Training of Neural Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Alvaro Velasquez, Susmit Jha, Ismail R. Alkhouri

The paper formalizes the concept of "dataless neural networks" (dNNs), which apply neural networks to optimization problems by optimizing model parameters against a single problem instance rather than requiring external training data. It offers a comprehensive survey, categorizing methods and distinguishing dNNs from related machine learning paradigms.

Key insights

  • Neural networks can effectively solve optimization problems by leveraging their inductive biases and differentiability, even without large training datasets.
  • The use of neural networks for optimization in a dataless setting has historical precedence and has seen renewed interest with modern NN architectures and computational resources.
  • Dataless neural networks fundamentally differ from zero-shot and one-shot learning by not relying on any form of pre-training on external datasets.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.