Reproduce “Phase Transitions in Learning Driven by Extremal Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ethan Levien, Trevor GrandPre
The authors analyze gradient descent in a Gaussian planted-instance multiple-instance regression model with learnable softmax pooling. They derive a gradient-flow description with distinct retrieval and freezing transitions: failure to retrieve can freeze direction learning, while freezing removes sensitivity to softmax sharpness. Synthetic results show loss plateaus across starting overlaps at d=150 and n=1808. These findings apply to this asymptotic model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.