Although a powerful and ubiquitous optimization method, the Stochastic Gradient Descent has fundamental structural limitations that make it unsuitable for some types of complex landscapes and Bayesian inference.

The Stochastic Gradient Langevin Dynamics fills the gap between optimization and random (Monte Carlo) sampling.