Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differential Privacy Analysis of Decentralized Gossip Averaging under Varying Threat Models

Antti Koskela, Tejas Kulkarni

TL;DR

This work addresses differential privacy guarantees in fully decentralized gossip-based averaging without a central aggregator, under varying threat models. It introduces a linear dynamical systems framework to track how privacy leakage propagates through iterative gossip updates and uses projected Gaussian mechanisms to quantify DP leakage. A key result is that, with secure neighbor summation, the squared sensitivity grows linearly with the number of rounds, leading to an asymptotic DP parameter growth of $O(T)$ and, for doubly stochastic graphs, an equivalent bound to central aggregation via $ rac{ ext{posterior}}{ ext{prior}}$-type limits (specifically $ rac{(igDelta_T(j o i))^2}{T} o rac{oldsymbol{ar{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{oldsymbol{ oldsymbol{oldsymbol{oldsymbol{oldsymbol{ oldsymbol{}}}}}}}}}}}}}}}$). It also connects adaptive and non-adaptive compositions through the matrix mechanism and validates predictions with simulations on synthetic graphs. Overall, the paper provides a principled, scalable framework for analyzing DP in decentralized learning and demonstrates that secure aggregation can recover central-like privacy guarantees in decentralized settings.

Abstract

Fully decentralized training of machine learning models offers significant advantages in scalability, robustness, and fault tolerance. However, achieving differential privacy (DP) guarantees in such settings is challenging due to the absence of a central aggregator and varying trust assumptions among nodes. We present a novel privacy analysis of decentralized gossip-based averaging algorithms with additive node-level noise, from arbitrary views of nodes in a graph and especially consider the averaging over nearest neighbors with secure summation and individual node-wise views. Our main contribution is a an analytical framework based on a linear systems formulation that accurately characterizes privacy leakage between nodes across different scenarios. In case the gossip averaging happens via secure summation, we show that the Rényi DP parameter growth is asymptotically $O(T)$, where $T$ is the number of training rounds, similarly as in the case of central aggregation.

Differential Privacy Analysis of Decentralized Gossip Averaging under Varying Threat Models

TL;DR

This work addresses differential privacy guarantees in fully decentralized gossip-based averaging without a central aggregator, under varying threat models. It introduces a linear dynamical systems framework to track how privacy leakage propagates through iterative gossip updates and uses projected Gaussian mechanisms to quantify DP leakage. A key result is that, with secure neighbor summation, the squared sensitivity grows linearly with the number of rounds, leading to an asymptotic DP parameter growth of and, for doubly stochastic graphs, an equivalent bound to central aggregation via -type limits (specifically ). It also connects adaptive and non-adaptive compositions through the matrix mechanism and validates predictions with simulations on synthetic graphs. Overall, the paper provides a principled, scalable framework for analyzing DP in decentralized learning and demonstrates that secure aggregation can recover central-like privacy guarantees in decentralized settings.

Abstract

Fully decentralized training of machine learning models offers significant advantages in scalability, robustness, and fault tolerance. However, achieving differential privacy (DP) guarantees in such settings is challenging due to the absence of a central aggregator and varying trust assumptions among nodes. We present a novel privacy analysis of decentralized gossip-based averaging algorithms with additive node-level noise, from arbitrary views of nodes in a graph and especially consider the averaging over nearest neighbors with secure summation and individual node-wise views. Our main contribution is a an analytical framework based on a linear systems formulation that accurately characterizes privacy leakage between nodes across different scenarios. In case the gossip averaging happens via secure summation, we show that the Rényi DP parameter growth is asymptotically , where is the number of training rounds, similarly as in the case of central aggregation.

Paper Structure

This paper contains 28 sections, 9 theorems, 70 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Background
  3. Differential Privacy
  4. Projected Gaussian Mechanism and Moore--Penrose Pseudoinverse
  5. Decentralized Learning and Network Differential Privacy
  6. Local Data and Learning Objective.
  7. Dataset Adjacency.
  8. Mechanism and Observations.
  9. Network Differential Privacy.
  10. Gossip-Based Averaging.
  11. Privacy Analysis
  12. Gossip Averaging as Discrete-Time Linear State-Space Dynamics
  13. Analysis of Non-Adaptive Compositions
  14. Accounting for Nodes' Knowledge of Injected Noise
  15. Adaptive Compositions and Connection to Matrix-Mechanism Accounting
  16. ...and 13 more sections

Key Result

Lemma 2

A mechanism $\mathcal{M}$ satisfies $(\epsilon,\delta)$-DP if and only if, $\max_{D \sim D'} H_{{\rm e}\space^\varepsilon}(\mathcal{M}(X)||\mathcal{M}(X'))\leq \delta$.

Figures (1)

  • Figure 1: Left: Erdős--Rényi graph $G(n,p)$ with $n=100$ and $p=0.2$, Right: gossip matrix of a preferential-attachment model with $n=200$ nodes, and the scaled sensitivity $(\Delta^T(j\to i))^2/T$ as a function of $T$. We consider separately the cases when random nodes $i$ and $j$ are neighbors and not.

Theorems & Definitions (21)

  • Definition 1: dwork_et_al_2006
  • Lemma 2: balle2018subsampling
  • Definition 3: Gaussian Mechanism
  • Lemma 4: dong2022gaussian
  • Definition 5: Compact SVD and Moore--Penrose Pseudoinverse
  • Lemma 6
  • Definition 2.1: Primitive Matrix
  • Lemma 7
  • Theorem 8
  • Theorem 9: Bellet et al., 2025, Thm. 5
  • ...and 11 more