Reproduce “Learning to Optimize” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ke Li, Jitendra Malik
Ke Li and Jitendra Malik introduce a reinforcement learning framework for automatically discovering optimization algorithms for unconstrained continuous problems. This approach yields algorithms that demonstrate faster convergence and/or achieve better final objective values compared to hand-engineered methods across convex and non-convex problems, notably outperforming baselines in neural network training.
Key insights
- Optimization algorithms can be effectively synthesized as learned policies within a reinforcement learning framework, eliminating the need for manual design.
- By incorporating historical objective values and gradients into the state representation, the learned optimizer can dynamically adapt its 'hyperparameters' (step size and direction) on the fly without user input.
- Training on a diverse set of objective functions from a specific class allows the autonomous optimizer to learn problem-specific structures and generalize efficiently to new, unseen instances.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.