Reproduce “Random Feature Amplification: Feature Learning and Generalization in Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Spencer Frei, Niladri S. Chatterji, Peter L. Bartlett
The authors analyze how gradient descent learns useful features in a two-layer ReLU network on a four-cluster XOR classification problem with noisy labels. Training the first-layer weights amplifies initially weak alignment with cluster directions, while the output signs remain fixed. With early stopping, test error is at most the label-noise rate plus a finite-sample term, under stated assumptions; the guarantee is specific to this model and distribution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.