Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tight Analysis of Decentralized SGD: A Markov Chain Perspective

Lucas Versini, Paul Mangold, Aymeric Dieuleveut

TL;DR

This work reframes Decentralized SGD as a Markov chain and shows it converges to a stationary distribution with a first-order bias that splits into a decentralization–heterogeneity term and a stochastic term. It derives explicit first-order expansions for the bias and variance, proving that the variance decreases with the number of clients and that topology affects only higher-order terms, yielding a linear speed-up without averaging. The authors provide non-asymptotic bounds and a Richardson–Romberg extrapolation method to cancel the leading bias, plus experiments validating the theory across various network topologies. Overall, the paper advances a precise, fixed-point view of DSGD under stochasticity and decentralization, with practical implications for speeding up decentralized training in less connected graphs.

Abstract

We propose a novel analysis of the Decentralized Stochastic Gradient Descent (DSGD) algorithm with constant step size, interpreting the iterates of the algorithm as a Markov chain. We show that DSGD converges to a stationary distribution, with its bias, to first order, decomposable into two components: one due to decentralization (growing with the graph's spectral gap and clients' heterogeneity) and one due to stochasticity. Remarkably, the variance of local parameters is, at the first-order, inversely proportional to the number of clients, regardless of the network topology and even when clients' iterates are not averaged at the end. As a consequence of our analysis, we obtain non-asymptotic convergence bounds for clients' local iterates, confirming that DSGD has linear speed-up in the number of clients, and that the network topology only impacts higher-order terms.

Tight Analysis of Decentralized SGD: A Markov Chain Perspective

TL;DR

This work reframes Decentralized SGD as a Markov chain and shows it converges to a stationary distribution with a first-order bias that splits into a decentralization–heterogeneity term and a stochastic term. It derives explicit first-order expansions for the bias and variance, proving that the variance decreases with the number of clients and that topology affects only higher-order terms, yielding a linear speed-up without averaging. The authors provide non-asymptotic bounds and a Richardson–Romberg extrapolation method to cancel the leading bias, plus experiments validating the theory across various network topologies. Overall, the paper advances a precise, fixed-point view of DSGD under stochasticity and decentralization, with practical implications for speeding up decentralized training in less connected graphs.

Abstract

We propose a novel analysis of the Decentralized Stochastic Gradient Descent (DSGD) algorithm with constant step size, interpreting the iterates of the algorithm as a Markov chain. We show that DSGD converges to a stationary distribution, with its bias, to first order, decomposable into two components: one due to decentralization (growing with the graph's spectral gap and clients' heterogeneity) and one due to stochasticity. Remarkably, the variance of local parameters is, at the first-order, inversely proportional to the number of clients, regardless of the network topology and even when clients' iterates are not averaged at the end. As a consequence of our analysis, we obtain non-asymptotic convergence bounds for clients' local iterates, confirming that DSGD has linear speed-up in the number of clients, and that the network topology only impacts higher-order terms.
Paper Structure (32 sections, 26 theorems, 173 equations, 2 figures, 1 algorithm)

This paper contains 32 sections, 26 theorems, 173 equations, 2 figures, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Notations.
  3. PROBLEM SETTING
  4. Decentralized Learning.
  5. Assumptions.
  6. DETERMINISTIC ALGORITHM
  7. Quadratic functions.
  8. General functions.
  9. STOCHASTIC ALGORITHM
  10. DSGD as a Markov Chain.
  11. Convergence to a Stationary Distribution.
  12. Variance of DSGD.
  13. Bias of DSGD.
  14. Non-Asymptotic Bounds
  15. RICHARDSON-ROMBERG FOR DECENTRALIZED LEARNING
  16. ...and 17 more sections

Key Result

Lemma 1

Assume assum:functions, assum:commmat, and $\gamma \le 1/L$. Then the sequence $(\Theta_{t})_t$ generated by eq:DGD converges to a vector $\Theta_{\mathrm{det}}$, independent of $\Theta_{0}$, which satisfies Moreover, for all $t \ge 0$,

Figures (2)

  • Figure 1: DSGD for heterogeneous (top row) and homogeneous (bottom row) clients, for various numbers of clients $m$ and communication graphs. Graphs are fully connected (left), four clusters sparsely connected (middle), and ring (right). Colored areas indicate variations (± standard deviation) obtained from $20$ independent runs.
  • Figure 2: $\frac{1}{m} \sum_{i=1}^m\left\|\theta_i^t-\theta^*\right\|$ for deterministic (left) and stochastic (right) DGD with step size $\gamma$, step size $\gamma/2$ and RR extrapolation.

Theorems & Definitions (50)

  • Lemma 1
  • Lemma 2
  • Proposition 1
  • Lemma 3: larsson2025unified
  • Corollary 1
  • Proposition 2: First-order bias expansion
  • Proposition 3: Convergence of DSGD
  • Proposition 4
  • Lemma 4
  • Proposition 5
  • ...and 40 more