Reproduce “Teacher Geometry Shapes Learnability in Teacher-Student Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen, Wulfram Gerstner, Johanni Brea
The paper studies how teacher-network geometry affects whether gradient-based training lets a student copy a known function. It compares one-hidden-layer teachers with diverse, standard, or similar hidden-unit directions and readout signs; success means final training loss below 10^-18. Same-width students succeed most often on maximally dissimilar teachers in almost all tested settings, a result about training fit rather than real-world performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.