Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Control Methods for Optimization

Jinniao Qiu

TL;DR

This work develops a unified stochastic control framework for global optimization in both Euclidean space $\mathbb{R}^d$ and the Wasserstein space $\mathcal{P}_2(\mathbb{R}^d)$. By introducing a regularized control cost, it derives tractable representations via the Hamilton–Jacobi–Bellman equation, Cole–Hopf transformation, and Feynman–Kac formula in finite dimensions, while formulating and approximating the master equation on Wasserstein space through $N$-particle systems. The authors prove quantitative convergence: in the Euclidean setting the optimal value converges to the global minimum at rate $O(\varepsilon\ln(1/\varepsilon))$, and in the measure-valued setting the finite-particle value converges to the global optimum with rate $O(1/N)+O(\varepsilon\ln(1/\varepsilon))$, uniformly in regularization. They also provide Monte Carlo-based numerical schemes and extensive experiments demonstrating convergence and scalability to high dimensions. Overall, the paper offers a principled probabilistic approach to global optimization with rigorous convergence guarantees and practical algorithms grounded in stochastic control and mean-field theory.

Abstract

In this work, we investigate a stochastic control framework for global optimization over both finite-dimensional Euclidean spaces and the Wasserstein space of probability measures. In the Euclidean setting, the original minimization problem is approximated by a family of regularized stochastic control problems; using dynamic programming, we analyze the associated Hamilton--Jacobi--Bellman equations and obtain tractable representations via the Cole--Hopf transform and the Feynman--Kac formula. For optimization over probability measures, we formulate a regularized mean-field control problem characterized by a master equation, and further approximate it by controlled $N$-particle systems. We establish that, as the regularization parameter tends to zero (and as the particle number tends to infinity for the optimization over probability measures), the value of the control problem converges to the global minimum of the original objective. Building on the resulting probabilistic representations, Monte Carlo-based numerical schemes are proposed and numerical experiments are reported to illustrate the practical performance of the methods and to support the theoretical convergence rates.

Stochastic Control Methods for Optimization

TL;DR

This work develops a unified stochastic control framework for global optimization in both Euclidean space and the Wasserstein space . By introducing a regularized control cost, it derives tractable representations via the Hamilton–Jacobi–Bellman equation, Cole–Hopf transformation, and Feynman–Kac formula in finite dimensions, while formulating and approximating the master equation on Wasserstein space through -particle systems. The authors prove quantitative convergence: in the Euclidean setting the optimal value converges to the global minimum at rate , and in the measure-valued setting the finite-particle value converges to the global optimum with rate , uniformly in regularization. They also provide Monte Carlo-based numerical schemes and extensive experiments demonstrating convergence and scalability to high dimensions. Overall, the paper offers a principled probabilistic approach to global optimization with rigorous convergence guarantees and practical algorithms grounded in stochastic control and mean-field theory.

Abstract

In this work, we investigate a stochastic control framework for global optimization over both finite-dimensional Euclidean spaces and the Wasserstein space of probability measures. In the Euclidean setting, the original minimization problem is approximated by a family of regularized stochastic control problems; using dynamic programming, we analyze the associated Hamilton--Jacobi--Bellman equations and obtain tractable representations via the Cole--Hopf transform and the Feynman--Kac formula. For optimization over probability measures, we formulate a regularized mean-field control problem characterized by a master equation, and further approximate it by controlled -particle systems. We establish that, as the regularization parameter tends to zero (and as the particle number tends to infinity for the optimization over probability measures), the value of the control problem converges to the global minimum of the original objective. Building on the resulting probabilistic representations, Monte Carlo-based numerical schemes are proposed and numerical experiments are reported to illustrate the practical performance of the methods and to support the theoretical convergence rates.
Paper Structure (21 sections, 13 theorems, 135 equations, 6 figures, 2 algorithms)

This paper contains 21 sections, 13 theorems, 135 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Optimization over finite-dimensional spaces: $\mathcal{X} = \mathbb{R}^d$
  3. Stochastic control formulations of problem \ref{['opt-prob']}
  4. Recall a standard resolution of the regularized stochastic control problem \ref{['reg-sto-control-prob']}
  5. Main result on convergence analysis as $\varepsilon\to 0$
  6. Optimization over probability measures: $\mathcal{X}= \mathcal{P}_2(\mathbb{R}^d)$
  7. The master equation and $N$-Particle Approximation
  8. Main result on convergence analysis as $\varepsilon\to 0$ and $N\to \infty$
  9. Literature, contributions, and organization of the paper
  10. Literature.
  11. Contribution.
  12. Organization of the paper.
  13. Optimization over finite-dimensional spaces: $\mathcal{X} = \mathbb{R}^d$
  14. A convergence analysis when $\varepsilon\to 0$
  15. Well-posedness of the stochastic control problems
  16. ...and 6 more sections

Key Result

Theorem 1.1

Under certain assumptions on $G$, for each $\xi \in \arg\min_{x\in\mathbb{R}^d}G(x)$, it holds that where the constant $C$ depends on dimension $d$, regularity of $G$, and $\mathbb E\left[|x_0-\xi|^2\right]$.

Figures (6)

  • Figure 1: Trajectories of 20 particles for the Xin-She Yang 4 function optimization. The particles converge to the global minimum at the origin.
  • Figure 2: Convergence of the Stochastic Control Method for the Xin-She Yang 4 function optimization. The left panel shows the value error against $-\varepsilon\ln(\varepsilon)$, while the right panel shows the value error against $\varepsilon$. The linear trend in the left panel provides a better fit, as indicated by the lower RMSE, corroborating the theoretical convergence rate established in Theorem \ref{['thm-convergence-2']}.
  • Figure 3: 2D Newtonian Swarm Optimization. Left: Initial concentrated distribution at the origin. Right: Final distribution approximating the uniform measure on the unit circle, the global minimizer of the Newtonian interaction energy.
  • Figure 4: Convergence Rate of Energy Level. The plot of energy $G(\mu_\varepsilon)$ versus $\frac{1}{N}$ shows a linear trend, confirming the $O(1/N)$ convergence rate predicted by Theorem \ref{['thm:value_convergence']}.
  • Figure 5: Regularization Error of value error. Left: Plot of value error versus $\varepsilon \ln(1/\varepsilon)$ shows a linear trend, supporting the $\mathcal{O}(\varepsilon \ln(1/\varepsilon))$ error estimate. Right: Plot of value error versus $\varepsilon$. The linear trend in the left panel provides a better fit, as indicated by the lower RMSE, corroborating the theoretical convergence rate established in Theorem \ref{['thm:regularization_error']}.
  • ...and 1 more figures

Theorems & Definitions (29)

  • Theorem 1.1
  • Theorem 1.2: Global Convergence of the Method
  • Lemma 2.1
  • proof
  • Theorem 2.2
  • Remark 2.1
  • proof
  • Corollary 2.3
  • proof
  • Corollary 2.4
  • ...and 19 more