Reproduce “CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Miria Feng, Zachary Frangella, Mert Pilanci
CRONOS makes convex two-layer ReLU training more scalable by replacing matrix factorization in ADMM with Nyström-preconditioned conjugate gradients. For deeper networks, CRONOS-AM alternates a convex final-layer subproblem with updates to earlier layers. CRONOS converges to the reformulation’s global minimum under an approximate-low-rank assumption. On four-layer MLP ImageNet, CRONOS-AM reaches 88.47% peak validation accuracy, below Shampoo’s 89.72%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.