Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of Multi-Level Markov Chain Monte Carlo Adaptive Stochastic Gradient Algorithms

Antoine Godichon-Baggioni, Gabriel Lang, Sylvain Le Corff, Julien Stoehr, Sobihan Surendran

TL;DR

The paper tackles the bias introduced by finite-time MCMC gradient estimators in stochastic optimization and proposes a multilevel Monte Carlo (MLMC) framework to reduce bias to $O(T_n^{-1})$ with only $O(\log T_n)$ expected cost per iteration. Building MLMC into adaptive gradient methods yields new MLMC variants of Adagrad and AMSGrad, for which the authors establish convergence rates of $O(n^{-1/2})$ up to logarithmic factors under bias/variance/moment controls. They provide a general convergence theory for MLMC-adaptive SGD, plus specialized results for Adagrad and AMSGrad, including a practical instantiation with Random Walk MCMC and MALA to verify assumptions. The approach is demonstrated on Importance Weighted Autoencoders (IWAE) trained on CIFAR-10, where MLMC-IWAE attains comparable iteration-wise performance to BR-IWAE but with greater sample efficiency per computational budget, illustrating the method’s potential for scalable, MCMC-based gradient estimation in deep learning tasks.

Abstract

Stochastic optimization in learning and inference often relies on Markov chain Monte Carlo (MCMC) to approximate gradients when exact computation is intractable. However, finite-time MCMC estimators are biased, and reducing this bias typically comes at a higher computational cost. We propose a multilevel Monte Carlo gradient estimator whose bias decays as $O(T_{n}^{-1} )$ while its expected computational cost grows only as $O(log T_n )$, where $T_n$ is the maximal truncation level at iteration n. Building on this approach, we introduce a multilevel MCMC framework for adaptive stochastic gradient methods, leading to new multilevel variants of Adagrad and AMSGrad algorithms. Under conditions controlling the estimator bias and its second and third moments, we establish a convergence rate of order $O(n^{-1/2} )$ up to logarithmic factors. Finally, we illustrate these results on Importance-Weighted Autoencoders trained with the proposed multilevel adaptive methods.

Convergence of Multi-Level Markov Chain Monte Carlo Adaptive Stochastic Gradient Algorithms

TL;DR

The paper tackles the bias introduced by finite-time MCMC gradient estimators in stochastic optimization and proposes a multilevel Monte Carlo (MLMC) framework to reduce bias to with only expected cost per iteration. Building MLMC into adaptive gradient methods yields new MLMC variants of Adagrad and AMSGrad, for which the authors establish convergence rates of up to logarithmic factors under bias/variance/moment controls. They provide a general convergence theory for MLMC-adaptive SGD, plus specialized results for Adagrad and AMSGrad, including a practical instantiation with Random Walk MCMC and MALA to verify assumptions. The approach is demonstrated on Importance Weighted Autoencoders (IWAE) trained on CIFAR-10, where MLMC-IWAE attains comparable iteration-wise performance to BR-IWAE but with greater sample efficiency per computational budget, illustrating the method’s potential for scalable, MCMC-based gradient estimation in deep learning tasks.

Abstract

Stochastic optimization in learning and inference often relies on Markov chain Monte Carlo (MCMC) to approximate gradients when exact computation is intractable. However, finite-time MCMC estimators are biased, and reducing this bias typically comes at a higher computational cost. We propose a multilevel Monte Carlo gradient estimator whose bias decays as while its expected computational cost grows only as , where is the maximal truncation level at iteration n. Building on this approach, we introduce a multilevel MCMC framework for adaptive stochastic gradient methods, leading to new multilevel variants of Adagrad and AMSGrad algorithms. Under conditions controlling the estimator bias and its second and third moments, we establish a convergence rate of order up to logarithmic factors. Finally, we illustrate these results on Importance-Weighted Autoencoders trained with the proposed multilevel adaptive methods.
Paper Structure (34 sections, 14 theorems, 145 equations, 4 figures, 3 algorithms)

This paper contains 34 sections, 14 theorems, 145 equations, 4 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Multi-level adaptive optimization
  3. Notations.
  4. Adaptive stochastic approximation.
  5. Adaptive algorithms with Multi-level Monte Carlo.
  6. Convergence results of MLMC-Adaptive stochastic gradient algorithms
  7. Convergence in the general preconditioned case
  8. Convergence bounds for Adagrad and AMSGrad
  9. AMSGrad.
  10. Verifying assumptions A\ref{['A-bias-var']} and A\ref{['A-moment-3']}
  11. Random Walk Metropolis-Hastings algorithm (RWMH).
  12. Metropolis-Adjusted Langevin Algorithm (MALA).
  13. Application of MLMC to Importance Weighted Autoencoders
  14. Experiments
  15. Discussion
  16. ...and 19 more sections

Key Result

Theorem 3.1

Assume that AA-smooth -- AA-bias-var and AA-eigen hold. For any $N \geq 1$, let $R$ be as defined in eq:def-r with $(\lambda_n)_{n\geq 1} = (\underline{\lambda}_n)_{n \geq 1}$. Then,

Figures (4)

  • Figure 1: Squared gradient norm $\|\nabla V(\theta_N)\|^2$ for BR-IWAE and MLMC-IWAE trained with AMSGrad on CIFAR-10, shown as a function of epochs (left) and computational budget (right). Both plots use the same scale and are displayed on a logarithmic scale for improved readability. Bold lines represent the mean over 5 independent runs.
  • Figure 2: Negative log-likelihood for BR-IWAE and MLMC-IWAE trained with AMSGrad on CIFAR-10, shown as a function of epochs (left) and computational budget (right). Bold lines represent the mean over 5 independent runs.
  • Figure 3: Squared gradient norm $\|\nabla V(\theta_N)\|^2$ for BR-IWAE and MLMC-IWAE trained with Adagrad on CIFAR-10, shown as a function of epochs (left) and computational budget (right). Bold lines represent the mean over 5 independent runs.
  • Figure 4: Negative log-likelihood for BR-IWAE and MLMC-IWAE trained with Adagrad on CIFAR-10, shown as a function of epochs (left) and computational budget (right). Bold lines represent the mean over 5 independent runs.

Theorems & Definitions (27)

  • Theorem 3.1
  • Corollary 3.2
  • Theorem 3.3
  • Corollary 3.4
  • Theorem 3.5
  • Corollary 3.6
  • Proposition 3.7
  • Lemma A.1
  • proof
  • Lemma A.2
  • ...and 17 more