Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence and concentration properties of constant step-size SGD through Markov chains

Ibrahim Merad, Stéphane Gaïffas

TL;DR

This work analyzes constant-step SGD for smooth μ-strongly convex objectives through a Markov-chain lens. It establishes geometric ergodicity in total variation and Wasserstein-2 convergence to a unique invariant distribution π_γ, and shows that gradient-noise concentration transfers to both the iterates and π_γ, yielding non-asymptotic high-probability bounds for the final iterate and for Polyak–Ruppert tail-averages in the linear-gradient case. The results are complemented by dimension-free bounds under stronger concentration assumptions and are illustrated via linear and logistic regression applications. The framework provides practical, non-asymptotic guarantees for stochastic optimization with constant step-sizes and opens avenues for extending concentration results to broader settings.

Abstract

We consider the optimization of a smooth and strongly convex objective using constant step-size stochastic gradient descent (SGD) and study its properties through the prism of Markov chains. We show that, for unbiased gradient estimates with mildly controlled variance, the iteration converges to an invariant distribution in total variation distance. We also establish this convergence in Wasserstein-2 distance under a relaxed assumption on the gradient noise distribution compared to previous work. Our analysis shows that the SGD iterates and their invariant limit distribution \emph{inherit} sub-Gaussian or sub-exponential concentration properties when these hold true for the gradient. This allows the derivation of high-confidence bounds for the final estimate. Finally, under such conditions in the linear case, we obtain a dimension-free deviation bound for the Polyak-Ruppert average of a tail sequence. All our results are non-asymptotic and their consequences are discussed through a few applications.

Convergence and concentration properties of constant step-size SGD through Markov chains

TL;DR

This work analyzes constant-step SGD for smooth μ-strongly convex objectives through a Markov-chain lens. It establishes geometric ergodicity in total variation and Wasserstein-2 convergence to a unique invariant distribution π_γ, and shows that gradient-noise concentration transfers to both the iterates and π_γ, yielding non-asymptotic high-probability bounds for the final iterate and for Polyak–Ruppert tail-averages in the linear-gradient case. The results are complemented by dimension-free bounds under stronger concentration assumptions and are illustrated via linear and logistic regression applications. The framework provides practical, non-asymptotic guarantees for stochastic optimization with constant step-sizes and opens avenues for extending concentration results to broader settings.

Abstract

We consider the optimization of a smooth and strongly convex objective using constant step-size stochastic gradient descent (SGD) and study its properties through the prism of Markov chains. We show that, for unbiased gradient estimates with mildly controlled variance, the iteration converges to an invariant distribution in total variation distance. We also establish this convergence in Wasserstein-2 distance under a relaxed assumption on the gradient noise distribution compared to previous work. Our analysis shows that the SGD iterates and their invariant limit distribution \emph{inherit} sub-Gaussian or sub-exponential concentration properties when these hold true for the gradient. This allows the derivation of high-confidence bounds for the final estimate. Finally, under such conditions in the linear case, we obtain a dimension-free deviation bound for the Polyak-Ruppert average of a tail sequence. All our results are non-asymptotic and their consequences are discussed through a few applications.
Paper Structure (40 sections, 19 theorems, 187 equations)

This paper contains 40 sections, 19 theorems, 187 equations.

Table of Contents

  1. Introduction
  2. Main contributions
  3. Related works
  4. SGD as a Markov chain.
  5. High probability bounds.
  6. Polyak-Ruppert averaging.
  7. Paper organization
  8. Setting and notations
  9. Markov Chain and Geometric Ergodicity
  10. Iteration and Invariant Distribution Properties
  11. Wasserstein Convergence
  12. Confidence bounds
  13. Final iterate concentration bounds
  14. Polyak-Ruppert averaging
  15. Applications
  16. ...and 25 more sections

Key Result

Theorem 1

Under Assumptions asm:smooth_strongconvex and asm:gradient, the Markov chain $(\theta_t)_{t\geq 0}$ defined by iteration eq:sgd_iteration with step-size admits a unique invariant measure $\pi_{\gamma}$ and converges geometrically to it. Namely, for any initial $\theta_0 \in \mathbb{R}^d,$ there exist $\rho < 1$ and $M < +\infty$ such that where $\delta_{\theta_0}$ is the Dirac measure located at

Theorems & Definitions (29)

  • Theorem 1
  • Proposition 1
  • Definition 1
  • Definition 2
  • Proposition 2
  • Lemma 1
  • Proposition 3
  • Proposition 4
  • Proposition 5
  • proof
  • ...and 19 more