Reproduce “Phase Transition from Random Features to Fully Trainable Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xin He, Huan Lao, Xiliang Lu, Shijun Zhang
The authors study two-layer networks that keep some hidden neurons fixed while training the incoming parameters of others and all output weights. Their approximation bound combines fixed features with trainable features sampled from a ridge-leverage distribution. For spherical biased-ReLU features in dimension five, about 4.1% trainable neurons capture 90% of the bound’s possible reduction, not absolute error or a general training threshold.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.