Reproduce “A Single-Loop, Constant-Batch First-Order Penalty Method for Stochastic Bilevel Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingyu Chen, Ming Yang, Quanqi Hu, Tianbao Yang
SICO presents a single-loop method for stochastic bilevel optimization that uses constant-size batches. It takes stochastic-gradient steps on two lower-level problems, projects one estimate to keep the tracks close, and averages the outer-gradient estimate. Under bounded-variance gradients, it guarantees ε-stationarity in O(κ¹¹ ε⁻⁶) oracle calls; with lower-level mean-square smoothness, O(κ⁹ ε⁻⁴), under stated assumptions and warm-up.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.