Revisiting Stochastic Approximation and Stochastic Gradient Descent
Rajeeva Laxman Karandikar, Bhamidi Visweswara Rao, Mathukumalli Vidyasagar
TL;DR
The paper addresses convergence proofs for stochastic approximation (SA) and stochastic gradient descent (SGD) under noisy measurements without assuming finite variance. It introduces GSLLN (Generalized Strong Law of Large Numbers) as a rate-dependent tool that decouples noise from the objective map and proves SA and SGD converge under weaker conditions than the classical ODE or martingale approaches. It also extends the framework to zero-order SGD using $2d$ function evaluations and derives the weakest yet sufficient conditions for GSLLN to ensure convergence. While expanding theoretical applicability, the authors acknowledge practical constraints and point to SPSA as a two-evaluation alternative, with ongoing work to reduce gradient-free evaluations for high-dimensional problems.
Abstract
In this paper, we introduce a new approach to proving the convergence of the Stochastic Approximation (SA) and the Stochastic Gradient Descent (SGD) algorithms. The new approach is based on a concept called GSLLN (Generalized Strong Law of Large Numbers), which extends the traditional SLLN. Using this concept, we provide sufficient conditions for convergence, which effectively decouple the properties of the function whose zero we are trying to find, from the properties of the measurement errors (noise sequence). The new approach provides an alternative to the two widely used approaches, namely the ODE approach and the martingale approach, and also permits a wider class of noise signals than either of the two known approaches. In particular, the ``noise'' or measurement error \textit{need not} have a finite second moment, and under suitable conditions, not even a finite mean. By adapting this method of proof, we also derive sufficient conditions for the convergence of zero-order SGD, wherein the stochastic gradient is computed using $2d$ function evaluations, but no gradient computations. The sufficient conditions derived here are the weakest to date, thus leading to a considerable expansion of the applicability of SA and SGD theory.