A library for differentiable nonlinear optimization
Theseus is an efficient application-agnostic library for building custom nonlinear optimization layers in PyTorch to support constructing various problems in robotics and vision as end-to-end differentiable architectures. See summary thread on 🐦 Twitter.
Theseus is an efficient application-agnostic library for building custom nonlinear optimization layers in PyTorch to support constructing various problems in robotics and vision as end-to-end differentiable architectures. See summary thread on 🐦 Twitter.
Get started with: pip install theseus-ai
Get started with: pip install theseus-ai
Differentiable nonlinear optimization provides a general scheme to encode inductive priors, as the objective function can be partly parameterized by neural models and partly with expert domain-specific differentiable models. The ability to compute gradients end-to-end is retained by differentiating through the optimizer which allows neural models to train on the final task loss, while also taking advantage of priors captured by the optimizer.
Differentiable nonlinear optimization provides a general scheme to encode inductive priors, as the objective function can be partly parameterized by neural models and partly with expert domain-specific differentiable models. The ability to compute gradients end-to-end is retained by differentiating through the optimizer which allows neural models to train on the final task loss, while also taking advantage of priors captured by the optimizer.
Application Agnostic Interface and Efficiency Based Design
Application Agnostic Interface and Efficiency Based Design
NeurIPS 2022 Talk
NeurIPS 2022 Talk
Simple example with Theseus. Robotics and Vision examples here.
Simple example with Theseus. Robotics and Vision examples here.
This example is fitting the curve y to a dataset of N observations (x,y)~D. This is modeled as an Objective with a single CostFunction that computes the residual y - v e^x. The Objective and the GaussNewton optimizer are encapsulated into a TheseusLayer. With Adam and MSE loss, x is learned by differentiating through the TheseusLayer.
This example is fitting the curve y to a dataset of N observations (x,y)~D. This is modeled as an Objective with a single CostFunction that computes the residual y - v e^x. The Objective and the GaussNewton optimizer are encapsulated into a TheseusLayer. With Adam and MSE loss, x is learned by differentiating through the TheseusLayer.
Performance and Evaluations
Performance and Evaluations
Sparse solvers are more compute and memory efficient compared to dense solvers
Sparse solvers are more compute and memory efficient compared to dense solvers
Forward pass of Theseus is up to 20x faster than Ceres on large batched problems
Forward pass of Theseus is up to 20x faster than Ceres on large batched problems
Implicit differentiation needs only constant compute and memory with increasing optimizer iterations while providing better gradients
Implicit differentiation needs only constant compute and memory with increasing optimizer iterations while providing better gradients
Team
Team
Citation
Citation
@article{pineda2022theseus,
title = {{Theseus: A Library for Differentiable Nonlinear Optimization}},
author = {Luis Pineda and Taosha Fan and Maurizio Monge and Shobha Venkataraman and Paloma Sodhi and Ricky TQ Chen and Joseph Ortiz and Daniel DeTone and Austin Wang and Stuart Anderson and Jing Dong and Brandon Amos and Mustafa Mukadam},
journal = {Advances in Neural Information Processing Systems},
year = {2022}
}