Optimal Transport for Machine Learners
Gabriel Peyré
TL;DR
Optimal Transport provides a rigorous framework for comparing probability measures and shaping how mass is moved between distributions. The notes articulate both Monge and Kantorovich formulations, their duals, and the regularized Sinkhorn approach, detailing how entropic regularization yields scalable algorithms with solid convergence guarantees. A key theme is the bridging of transports with gradient flows, Benamou–Brenier dynamics, and flow-based generative modelling, revealing OT’s role in quantifying and guiding distributional shaping. The work emphasizes mathematical structure (duality, TV, W1, W2, Bures metric on Gaussians) while highlighting practical numerical methods (Hungarian, Sinkhorn, Laguerre cells) and implications for ML tasks including diffusion models and transformer token dynamics.
Abstract
Optimal Transport is a foundational mathematical theory that connects optimization, partial differential equations, and probability. It offers a powerful framework for comparing probability distributions and has recently become an important tool in machine learning, especially for designing and evaluating generative models. These course notes cover the fundamental mathematical aspects of OT, including the Monge and Kantorovich formulations, Brenier's theorem, the dual and dynamic formulations, the Bures metric on Gaussian distributions, and gradient flows. It also introduces numerical methods such as linear programming, semi-discrete solvers, and entropic regularization. Applications in machine learning include topics like training neural networks via gradient flows, token dynamics in transformers, and the structure of GANs and diffusion models. These notes focus primarily on mathematical content rather than deep learning techniques.