Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sharp Gaussian approximations for Decentralized Federated Learning

Soham Bonnerjee, Sayar Karmakar, Wei Biao Wu

TL;DR

This work advances statistical inference for decentralized federated learning by establishing Berry-Esseen bounds for local-SGD and deriving time-uniform Gaussian couplings over the full learning trajectory. It introduces two Gaussian-approximation schemes, Aggr-GA (aggregated) and Client-GA (client-level), enabling sharper bootstrap-based inference and robust attack detection in distributed settings. The results reveal a computation-communication trade-off and a phase-transition in inference quality as the number of clients $K$ grows with the total iterations $n$, supported by comprehensive simulations and attack-detection experiments (including MNIST). Together, these contributions provide practical tools for valid inference, anomaly detection, and privacy-preserving diagnostics in decentralized learning pipelines. The framework paves the way for broader applications in multi-agent systems and secure distributed optimization.

Abstract

Federated Learning has gained traction in privacy-sensitive collaborative environments, with local SGD emerging as a key optimization method in decentralized settings. While its convergence properties are well-studied, asymptotic statistical guarantees beyond convergence remain limited. In this paper, we present two generalized Gaussian approximation results for local SGD and explore their implications. First, we prove a Berry-Esseen theorem for the final local SGD iterates, enabling valid multiplier bootstrap procedures. Second, motivated by robustness considerations, we introduce two distinct time-uniform Gaussian approximations for the entire trajectory of local SGD. The time-uniform approximations support Gaussian bootstrap-based tests for detecting adversarial attacks. Extensive simulations are provided to support our theoretical results.

Sharp Gaussian approximations for Decentralized Federated Learning

TL;DR

This work advances statistical inference for decentralized federated learning by establishing Berry-Esseen bounds for local-SGD and deriving time-uniform Gaussian couplings over the full learning trajectory. It introduces two Gaussian-approximation schemes, Aggr-GA (aggregated) and Client-GA (client-level), enabling sharper bootstrap-based inference and robust attack detection in distributed settings. The results reveal a computation-communication trade-off and a phase-transition in inference quality as the number of clients grows with the total iterations , supported by comprehensive simulations and attack-detection experiments (including MNIST). Together, these contributions provide practical tools for valid inference, anomaly detection, and privacy-preserving diagnostics in decentralized learning pipelines. The framework paves the way for broader applications in multi-agent systems and secure distributed optimization.

Abstract

Federated Learning has gained traction in privacy-sensitive collaborative environments, with local SGD emerging as a key optimization method in decentralized settings. While its convergence properties are well-studied, asymptotic statistical guarantees beyond convergence remain limited. In this paper, we present two generalized Gaussian approximation results for local SGD and explore their implications. First, we prove a Berry-Esseen theorem for the final local SGD iterates, enabling valid multiplier bootstrap procedures. Second, motivated by robustness considerations, we introduce two distinct time-uniform Gaussian approximations for the entire trajectory of local SGD. The time-uniform approximations support Gaussian bootstrap-based tests for detecting adversarial attacks. Extensive simulations are provided to support our theoretical results.
Paper Structure (60 sections, 18 theorems, 138 equations, 4 figures, 5 tables, 2 algorithms)

This paper contains 60 sections, 18 theorems, 138 equations, 4 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Notations
  4. Related Literature
  5. Berry-Esseen theory for local SGD
  6. Preliminaries
  7. Communication
  8. Berry-Esseen theorems for client-averaged local SGD updates
  9. Estimating Sigma_n
  10. A time-uniform Gaussian coupling for the DFL updates
  11. Motivation and Applications
  12. Optimal coupling for local SGD
  13. Simulation results
  14. Effect of n and K on the Berry-Esseen rate
  15. Computation-communication trade-off
  16. ...and 45 more sections

Key Result

Theorem 1.1

For a decentralized federated learning set-up with $K$ clients, the Polyak-ruppert averaged iterates of the local SGD algorithm with $n$ iterations, and step size $\eta_t \asymp t^{-\beta}$, achieves

Figures (4)

  • Figure 1: Plot of $\tilde{d}_c$ against $n$ and $K$ for $\gamma=1$, and Settings 1(left), and 2(right).
  • Figure 2: Plot of ${d}_c^\dagger$ against $(n, \beta)$ for $r=0.2$ (left), and $r=0.6$ (right). Here $\gamma=0$.
  • Figure 3: QQ-plots of $U_n^\texttt{Aggr-GA}$ (blue), $U_n^\texttt{Client-GA}$ (green) and $U_n^\texttt{f-CLT}$ (orange) against $U_n$ for $\gamma=1$, $N=500, \tau=20$. Here $K=10$(left), $K=25$(middle), $K=50$(right). Rest of the FRand-eff model specifications are as in Section \ref{['se:randeffmodel']}.
  • Figure 4: Plot of $\tilde{d}_c$ against $n$ and $K$ for $\gamma=5$, and Settings 1(left), and 2(right).

Theorems & Definitions (29)

  • Theorem 1.1: Theorem \ref{['thm: berry-esseen']}, Informal
  • Theorem 1.2: Theorem \ref{['thm: kmt_dfl']}, Informal
  • Theorem 2.1
  • Remark 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Remark 2.2: Computation-communication trade-off
  • Theorem 3.1
  • Theorem 3.2
  • Remark 3.1: Computational differences between Aggr-GA and Client-GA
  • ...and 19 more