Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Source Anonymity for Private Random Walk Decentralized Learning

Maximilian Egger, Svenja Lage, Rawad Bitar, Antonia Wachter-Zeh

TL;DR

This work tackles privacy in random-walk based decentralized learning by conceiving a source-anonymity framework that hides the updater's identity while allowing the destination to decrypt and incorporate updates. It introduces a formal $\alpha$-privacy notion based on entropy over candidate sources, and uses public-key cryptography to ensure updates are decryptable only by the intended destination. Focusing on Random Regular Graphs, it derives optimal distance-based destination distributions that render the source appears uniform to any destination, and extends the analysis to scenarios with side information by incorporating a hitting-time cap $\kappa$ and deriving probabilistic guarantees. Numerical experiments on large RRGs confirm the approach yields near-optimal anonymity with controllable runtime overhead, illustrating a practical privacy-utility trade-off without resorting to differential privacy.

Abstract

This paper considers random walk-based decentralized learning, where at each iteration of the learning process, one user updates the model and sends it to a randomly chosen neighbor until a convergence criterion is met. Preserving data privacy is a central concern and open problem in decentralized learning. We propose a privacy-preserving algorithm based on public-key cryptography and anonymization. In this algorithm, the user updates the model and encrypts the result using a distant user's public key. The encrypted result is then transmitted through the network with the goal of reaching that specific user. The key idea is to hide the source's identity so that, when the destination user decrypts the result, it does not know who the source was. The challenge is to design a network-dependent probability distribution (at the source) over the potential destinations such that, from the receiver's perspective, all users have a similar likelihood of being the source. We introduce the problem and construct a scheme that provides anonymity with theoretical guarantees. We focus on random regular graphs to establish rigorous guarantees.

Source Anonymity for Private Random Walk Decentralized Learning

TL;DR

This work tackles privacy in random-walk based decentralized learning by conceiving a source-anonymity framework that hides the updater's identity while allowing the destination to decrypt and incorporate updates. It introduces a formal -privacy notion based on entropy over candidate sources, and uses public-key cryptography to ensure updates are decryptable only by the intended destination. Focusing on Random Regular Graphs, it derives optimal distance-based destination distributions that render the source appears uniform to any destination, and extends the analysis to scenarios with side information by incorporating a hitting-time cap and deriving probabilistic guarantees. Numerical experiments on large RRGs confirm the approach yields near-optimal anonymity with controllable runtime overhead, illustrating a practical privacy-utility trade-off without resorting to differential privacy.

Abstract

This paper considers random walk-based decentralized learning, where at each iteration of the learning process, one user updates the model and sends it to a randomly chosen neighbor until a convergence criterion is met. Preserving data privacy is a central concern and open problem in decentralized learning. We propose a privacy-preserving algorithm based on public-key cryptography and anonymization. In this algorithm, the user updates the model and encrypts the result using a distant user's public key. The encrypted result is then transmitted through the network with the goal of reaching that specific user. The key idea is to hide the source's identity so that, when the destination user decrypts the result, it does not know who the source was. The challenge is to design a network-dependent probability distribution (at the source) over the potential destinations such that, from the receiver's perspective, all users have a similar likelihood of being the source. We introduce the problem and construct a scheme that provides anonymity with theoretical guarantees. We focus on random regular graphs to establish rigorous guarantees.
Paper Structure (15 sections, 9 theorems, 56 equations, 3 figures)

This paper contains 15 sections, 9 theorems, 56 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Model and Anonymity Notion
  3. Optimal Source Anonymity
  4. On the First Hitting Time of RRGs
  5. Source Anonymity in Light of RRGs
  6. Source Anonymity under Side Information
  7. Numerical Evaluations
  8. Conclusion
  9. Proof of \ref{['lem:unnormalized_likelihood_sp']} and \ref{['lem:unnormalized_likelihood']}
  10. Proof of \ref{['lemma:rrgoptprelim_sp']} and \ref{['lemma:rrgopt']}
  11. Proof of \ref{['prop:truncexpfht']}
  12. Proof of \ref{['thm:tvboundest']}
  13. Proof of \ref{['thm:estentropybound']}
  14. Proof of \ref{['thm:optimalsol_entropybound']}
  15. Privacy-Utility Trade-Off

Key Result

Lemma 1

Let $i,j\in \mathcal{V}$ be two nodes within distance $\ell\in[\ell_1,\ell_2]$ and let $\delta>0$ be fixed. Under ass:fht, the integral in (Gl2) is given by where $E_{\delta}(\ell) = K_{\delta}(\ell) e^{-c^\prime\frac{\mathbb{E}[T_\text{FH}\vert\ell]}{N}}$.

Figures (3)

  • Figure 1: RW-model for $N=300$ nodes and degree $c=3$ optimized for $\kappa=634$ and $\delta=5$. We compare the entropies over varying side information $\kappa^\prime$, with and without \ref{['ass:fht']}, compared with no countermeasure and the theoretical guarantee from \ref{['thm:optimalsol_entropybound']} using $\delta^\prime=0.3$. Uniform distribution supported on $[\ell_1, \ell_2] = [2,6]$ has entropy $\log(\ell_2-\ell_1+1)\approx 1.61$.
  • Figure 2: $\alpha$-privacy over probabilistic guarantee for $N=300$, $c=3$ and $\delta=5$ with and without \ref{['ass:fht']}, compared with no countermeasure and the theoretical guarantee from \ref{['thm:optimalsol_entropybound']}.
  • Figure 3: Entropy over the average iteration times $T_{\ell_1, \ell_2}$ for $\kappa^\prime=\kappa$ as chosen in \ref{['sec:experiments']}, $N=300$ and $c=4$.

Theorems & Definitions (17)

  • Definition 1
  • Lemma 1
  • Lemma 2
  • Remark 1
  • Proposition 1
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • ...and 7 more