Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations

Martino Bardi, Hicham Kouhkouh

TL;DR

A singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. is considered to approximate the Entropic Gradient Descent in the optimization of deep neural networks, via homogenisation, and it is shown that the limit of the value functions is itself the value function of an effective control problem with extended controls.

Abstract

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the Entropic Gradient Descent in the optimization of deep neural networks, via homogenisation. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton-Jacobi-Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls, and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modelled as dynamic controls.

Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations

TL;DR

A singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. is considered to approximate the Entropic Gradient Descent in the optimization of deep neural networks, via homogenisation, and it is shown that the limit of the value functions is itself the value function of an effective control problem with extended controls.

Abstract

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the Entropic Gradient Descent in the optimization of deep neural networks, via homogenisation. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton-Jacobi-Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls, and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modelled as dynamic controls.
Paper Structure (14 sections, 16 theorems, 139 equations)

This paper contains 14 sections, 16 theorems, 139 equations.

Table of Contents

  1. Introduction
  2. Local entropy and deep relaxation
  3. Deep relaxation with control of the learning rate
  4. General singular perturbations in stochastic control
  5. The two scale stochastic control problem
  6. The system
  7. The optimal control problem and HJB equation
  8. The limit PDE and convergence of the value functions
  9. The effective control problem
  10. A control interpretation of the limit PDE
  11. The limit $V$ is a value function
  12. Convergence of the trajectories
  13. Convergence of trajectories with vanishing diffusion
  14. The model problem: deep relaxation with learning rate

Key Result

Corollary 1.1

Let $\phi\in C^{1}(\mathds{R}^{n})$ with $\nabla\phi$ Lipschitz continuous. Then, for all $T>0$, for $\gamma$ in eq: V in application small enough, (i) for any $y\in \mathds{R}^{n}$ the $x$-component of the trajectory $(X^{\varepsilon}, Y^{\varepsilon})$ of homo_sys converges to the solution of lim (ii) if for a sequence of processes $(X^{\varepsilon_{n}}, Y^{\varepsilon_{n}})$ solving homo_sys

Theorems & Definitions (36)

  • Corollary 1.1
  • Theorem 1.1
  • Corollary 1.2
  • Proposition 2.1
  • Proposition 2.2
  • proof
  • Proposition 2.3
  • proof
  • Theorem 2.1
  • proof
  • ...and 26 more