Reproduce “A Mathematical Model for Curriculum Learning for Parities” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Elisabetta Cornacchia, Elchanan Mossel
Researchers from EPFL and MIT developed a mathematical model for curriculum learning (CL), demonstrating that a two-step CL strategy allows neural networks to learn k-parities in polynomial time, a task otherwise exponentially hard. Their work also introduces "Hamming mixtures," functions that prove difficult for discrete CL, suggesting limitations of current curriculum designs.
Key insights
- Curriculum learning can theoretically overcome computational hardness by structuring the learning process from 'easier' biased distributions to 'harder' uniform distributions.
- The success of the 2-CL strategy for k-parities hinges on decoupling the problem into support recovery (achieved through biased training) and subsequent generalization (achieved through uniform training).
- Bounded-step curriculum learning strategies face fundamental limitations for functions like 'Hamming mixtures,' which are locally simple but globally complex, suggesting the need for continuous or more adaptive curricula.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.