Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Steady-State Behavior of Constant-Stepsize Stochastic Approximation: Gaussian Approximation and Tail Bounds

Zedong Wang, Yuyang Wang, Ijay Narang, Felix Wang, Yuzhou Wang, Siva Theja Maguluri

TL;DR

This paper provides explicit, non-asymptotic error bounds for fixed $\alpha$ and identifies a non-Gaussian (Gibbs) limiting law under the correct scaling, which is validated numerically, and provides a corresponding pre-limit Wasserstein error bound.

Abstract

Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency. For a fixed stepsize, the iterates typically admit a stationary distribution that is rarely tractable. Prior work shows that as the stepsize $α\downarrow 0$, the centered-and-scaled steady state converges weakly to a Gaussian random vector. However, for fixed $α$, this weak convergence offers no usable error bound for approximating the steady-state by its Gaussian limit. This paper provides explicit, non-asymptotic error bounds for fixed $α$. We first prove general-purpose theorems that bound the Wasserstein distance between the centered-scaled steady state and an appropriate Gaussian distribution, under regularity conditions for drift and moment conditions for noise. To ensure broad applicability, we cover both i.i.d. and Markovian noise models. We then instantiate these theorems for three representative SA settings: (1) stochastic gradient descent (SGD) for smooth strongly convex objectives, (2) linear SA, and (3) contractive nonlinear SA. We obtain dimension- and stepsize-dependent, explicit bounds in Wasserstein distance of order $α^{1/2}\log(1/α)$ for small $α$. Building on the Wasserstein approximation error, we further derive non-uniform Berry--Esseen-type tail bounds that compare the steady-state tail probability to Gaussian tails. We achieve an explicit error term that decays in both the deviation level and stepsize $α$. We adapt the same analysis for SGD beyond strongly convexity and study general convex objectives. We identify a non-Gaussian (Gibbs) limiting law under the correct scaling, which is validated numerically, and provide a corresponding pre-limit Wasserstein error bound.

Steady-State Behavior of Constant-Stepsize Stochastic Approximation: Gaussian Approximation and Tail Bounds

TL;DR

This paper provides explicit, non-asymptotic error bounds for fixed and identifies a non-Gaussian (Gibbs) limiting law under the correct scaling, which is validated numerically, and provides a corresponding pre-limit Wasserstein error bound.

Abstract

Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency. For a fixed stepsize, the iterates typically admit a stationary distribution that is rarely tractable. Prior work shows that as the stepsize , the centered-and-scaled steady state converges weakly to a Gaussian random vector. However, for fixed , this weak convergence offers no usable error bound for approximating the steady-state by its Gaussian limit. This paper provides explicit, non-asymptotic error bounds for fixed . We first prove general-purpose theorems that bound the Wasserstein distance between the centered-scaled steady state and an appropriate Gaussian distribution, under regularity conditions for drift and moment conditions for noise. To ensure broad applicability, we cover both i.i.d. and Markovian noise models. We then instantiate these theorems for three representative SA settings: (1) stochastic gradient descent (SGD) for smooth strongly convex objectives, (2) linear SA, and (3) contractive nonlinear SA. We obtain dimension- and stepsize-dependent, explicit bounds in Wasserstein distance of order for small . Building on the Wasserstein approximation error, we further derive non-uniform Berry--Esseen-type tail bounds that compare the steady-state tail probability to Gaussian tails. We achieve an explicit error term that decays in both the deviation level and stepsize . We adapt the same analysis for SGD beyond strongly convexity and study general convex objectives. We identify a non-Gaussian (Gibbs) limiting law under the correct scaling, which is validated numerically, and provide a corresponding pre-limit Wasserstein error bound.
Paper Structure (73 sections, 19 theorems, 345 equations, 4 figures)

This paper contains 73 sections, 19 theorems, 345 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Our Approach
  4. Problem Setup
  5. I.I.D. Noise Case
  6. General Purpose Theorem under i.i.d. Noise
  7. Three Application Models
  8. SGD for Strongly Convex and Smooth Objective
  9. Linear SA
  10. SA with Contractive Nonlinear Operator
  11. Markovian Noise Setting
  12. SGD for general objectives and Gibbs limit distribution
  13. Future Work
  14. Structure of the Appendix
  15. Related Literature
  16. ...and 58 more sections

Key Result

Theorem 3.1

Consider the constant-stepsize SA recursion where $X_0^{(\alpha)}\in\mathbb{R}^d$ and $\alpha>0$ is fixed. We require that there exists $\alpha_0\in(0,1]$ such that for all $\alpha\in(0,\alpha_0)$, the following conditions hold: Define the target Gaussian distribution $Y\sim\mathcal{N}(0,\Sigma_Y)$. Then there exist $\alpha_1\in(0,1]$ and uniform constant $U\in(0,\infty)$, such that for all $\al

Figures (4)

  • Figure 1: Fluctuation theory and distributional approximation diagram.
  • Figure 2: Basline versus scaling for Normal Noise
  • Figure 4: Combined Results for Polynomial and Trigonometric Functions
  • Figure 5: SA Iterations and Concentration Bound

Theorems & Definitions (32)

  • Theorem 3.1: i.i.d. Gaussian approximation
  • Lemma 3.1
  • Proposition 3.1
  • Proposition 3.2
  • Lemma 3.2
  • Proposition 3.3
  • Definition 4.1: Long-Run Covariance and Poisson Equations
  • Theorem 4.1: Markovian Gaussian approximation
  • Proposition 4.1
  • Conjecture 5.1
  • ...and 22 more