Reproduce “Theseus: A Library for Differentiable Nonlinear Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Luis Pineda, Taosha Fan, Maurizio Monge, Shobha Venkataraman, Paloma Sodhi, Ricky T. Q. Chen, Joseph Ortiz, Daniel DeTone and 5 more
Meta AI and Reality Labs Research developed Theseus, an open-source PyTorch library for differentiable nonlinear least squares (DNLS) optimization, which enables efficient end-to-end learning by bridging classical optimization with deep learning. The library, featuring a novel batched sparse Cholesky solver (BaSpaCho), achieves up to 23x faster performance than the state-of-the-art Ceres Solver for large batched problems.
Key insights
- Seamlessly integrating traditional nonlinear optimization as a differentiable layer within deep learning frameworks enables more powerful end-to-end structured learning.
- Custom batched and GPU-accelerated sparse linear solvers are essential for achieving high performance and scalability in differentiable nonlinear least squares, especially for large problem batches.
- Efficient backward modes, such as implicit differentiation, overcome the computational and memory limitations of unrolling for second-order optimization layers, making them practical for deep learning integration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.