Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updating

TL;DR

This work broadens the convergence theory for the stochastic Heavy Ball method by allowing biased stochastic gradients with variance that can grow over time, and by permitting random step sizes suitable for block updating. By reframing SHB as SGD in an enlarged state space and employing a Lyapunov-like analysis with Robbins-Siegmund arguments, the authors prove almost-sure convergence under stochastic Robbins-Monro-type conditions and establish results for PL and KL' objective functions, including rate statements under PL. A meta-theorem extends these results to block updating schemes, ensuring convergence when only subsets of coordinates are updated. The paper also demonstrates that SHB can be effective with zero-order gradient estimates (gradient-free methods) and provides numerical evidence comparing SHB to other optimizers under noisy and approximate gradient regimes, highlighting SHB’s robustness in gradient-free settings.

Abstract

In this paper, we establish the convergence of the stochastic Heavy Ball (SHB) algorithm under more general conditions than in the current literature. Specifically, (i) The stochastic gradient is permitted to be biased, and also, to have conditional variance that grows over time (or iteration number). This feature is essential when applying SHB with zeroth-order methods, which use only two function evaluations to approximate the gradient. In contrast, all existing papers assume that the stochastic gradient is unbiased and/or has bounded conditional variance. (ii) The step sizes are permitted to be random, which is essential when applying SHB with block updating. The sufficient conditions for convergence are stochastic analogs of the well-known Robbins-Monro conditions. This is in contrast to existing papers where more restrictive conditions are imposed on the step size sequence. (iii) Our analysis embraces not only convex functions, but also more general functions that satisfy the PL (Polyak-Łojasiewicz) and KL (Kurdyka-Łojasiewicz) conditions. (iv) If the stochastic gradient is unbiased and has bounded variance, and the objective function satisfies (PL), then the iterations of SHB match the known best rates for convex functions. (v) We establish the almost-sure convergence of the iterations, as opposed to convergence in the mean or convergence in probability, which is the case in much of the literature. (vi) Each of the above convergence results continue to hold if full-coordinate updating is replaced by any one of three widely-used updating methods. In addition, numerical computations are carried out to illustrate the above points.

Figures (4)

• Figure 1: Comparison of various algorithms with noisy gradients (true gradients corrupted by additive zero mean Gaussian noise)
• Figure 2: Comparison of various algorithms with approximate gradients (gradients approximated using \ref{['eq:453']})
• Figure 3: Comparison of various algorithms with noisy gradients and block updating, with various choices of $\rho$
• Figure 4: Comparison of various algorithms with approximate gradients and block updating, with various choices of $\rho$

Theorems & Definitions (18)

• Lemma 1
• Lemma 2
• Lemma 3
• Theorem 1
• Definition 1
• Theorem 2
• Theorem 3
• Theorem 4
• Theorem 5
• Proof 1