Reproduce “Meta-Learning Strategies through Value Maximization in Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rodrigo Carrasco-Davis, Javier Masís, Andrew M. Saxe
The authors formulate meta-learning as choosing time-varying control signals to alter training and optimize discounted performance minus control cost. They predict learning with tractable gradient-descent dynamics, applying the framework to multi-step MAML and task or class engagement. In a two-layer linear network on five MNIST digit-pair tasks, longer-horizon optimization improved trajectories over 8,000 updates, but only in that tested setting.
Key insights
- Optimal control effort consistently prioritizes easier aspects of a task early in learning, followed by sustained effort on harder aspects, leading to improved overall learning speed and cumulative performance.
- Optimizing meta-strategies for multiple future gradient steps (e.g., in Multi-Step MAML) significantly outperforms myopic (one-step) approaches by accounting for long-term learning dynamics.
- Insights gained from tractable deep linear network models, specifically regarding optimal control allocation, can be relevant and provide performance improvements when applied to approximated non-linear networks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.