Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Topology-Based Reconstruction Prevention for Decentralised Learning

Florine W. Dekker, Zekeriya Erkin, Mauro Conti

TL;DR

This work analyzes reconstruction attacks in privacy-preserving multi-party summation for decentralised learning and shows that adversaries can reconstruct private data using local topology, even without auxiliary knowledge. It introduces a topology-based defence based on increasing the graph girth, proving that reconstruction requires cycles and that a girth larger than $2k$ (where $k$ is the number of colluding adversaries) prevents exact partial solutions; dynamic graphs can be reduced to static representations for the analysis. Empirical results on random subgraphs reveal non-negligible reconstruction risk (e.g., $11.0\%$ to reconstruct at least one datum with 3 adversaries and 15 neighbours, averaging $8.8$ rounds to success), highlighting the trade-off with convergence speed in higher-girth networks. The paper lays groundwork for a formal topology-aware confidentiality theory and suggests future work on necessary conditions, broader operations beyond summation, and integration with differential privacy noise.

Abstract

Decentralised learning has recently gained traction as an alternative to federated learning in which both data and coordination are distributed. To preserve the confidentiality of users' data, decentralised learning relies on differential privacy, multi-party computation, or both. However, running multiple privacy-preserving summations in sequence may allow adversaries to perform reconstruction attacks. Current reconstruction countermeasures either cannot trivially be adapted to the distributed setting, or add excessive amounts of noise. In this work, we first show that passive honest-but-curious adversaries can infer other users' private data after several privacy-preserving summations. For example, in subgraphs with 18 users, we show that only three passive honest-but-curious adversaries succeed at reconstructing private data 11.0% of the time, requiring an average of 8.8 summations per adversary. The success rate depends only on the adversaries' direct neighbourhood, and is independent of the size of the full network. We consider weak adversaries that do not control the graph topology, cannot exploit the summation's inner workings, and do not have auxiliary knowledge; and show that these adversaries can still infer private data. We analyse how reconstruction relates to topology and propose the first topology-based decentralised defence against reconstruction attacks. We show that reconstruction requires a number of adversaries linear in the length of the network's shortest cycle. Consequently, exact attacks over privacy-preserving summations are impossible in acyclic networks. Our work is a stepping stone for a formal theory of topology-based decentralised reconstruction defences. Such a theory would generalise our countermeasure beyond summation, define confidentiality in terms of entropy, and describe the interactions with (topology-aware) differential privacy.

Topology-Based Reconstruction Prevention for Decentralised Learning

TL;DR

This work analyzes reconstruction attacks in privacy-preserving multi-party summation for decentralised learning and shows that adversaries can reconstruct private data using local topology, even without auxiliary knowledge. It introduces a topology-based defence based on increasing the graph girth, proving that reconstruction requires cycles and that a girth larger than (where is the number of colluding adversaries) prevents exact partial solutions; dynamic graphs can be reduced to static representations for the analysis. Empirical results on random subgraphs reveal non-negligible reconstruction risk (e.g., to reconstruct at least one datum with 3 adversaries and 15 neighbours, averaging rounds to success), highlighting the trade-off with convergence speed in higher-girth networks. The paper lays groundwork for a formal topology-aware confidentiality theory and suggests future work on necessary conditions, broader operations beyond summation, and integration with differential privacy noise.

Abstract

Decentralised learning has recently gained traction as an alternative to federated learning in which both data and coordination are distributed. To preserve the confidentiality of users' data, decentralised learning relies on differential privacy, multi-party computation, or both. However, running multiple privacy-preserving summations in sequence may allow adversaries to perform reconstruction attacks. Current reconstruction countermeasures either cannot trivially be adapted to the distributed setting, or add excessive amounts of noise. In this work, we first show that passive honest-but-curious adversaries can infer other users' private data after several privacy-preserving summations. For example, in subgraphs with 18 users, we show that only three passive honest-but-curious adversaries succeed at reconstructing private data 11.0% of the time, requiring an average of 8.8 summations per adversary. The success rate depends only on the adversaries' direct neighbourhood, and is independent of the size of the full network. We consider weak adversaries that do not control the graph topology, cannot exploit the summation's inner workings, and do not have auxiliary knowledge; and show that these adversaries can still infer private data. We analyse how reconstruction relates to topology and propose the first topology-based decentralised defence against reconstruction attacks. We show that reconstruction requires a number of adversaries linear in the length of the network's shortest cycle. Consequently, exact attacks over privacy-preserving summations are impossible in acyclic networks. Our work is a stepping stone for a formal theory of topology-based decentralised reconstruction defences. Such a theory would generalise our countermeasure beyond summation, define confidentiality in terms of entropy, and describe the interactions with (topology-aware) differential privacy.
Paper Structure (32 sections, 7 theorems, 14 equations, 7 figures)

This paper contains 32 sections, 7 theorems, 14 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reconstruction Attacks
  4. Multi-Party Computation
  5. Preliminaries
  6. Privacy-Preserving Summation
  7. Bipartite Graphs
  8. Assumptions and Notation
  9. User data and objectives
  10. Network model
  11. Adversarial model
  12. Time model
  13. Reconstruction in Multi-party Summation
  14. Introduction to Reconstruction Attacks
  15. A small example
  16. ...and 17 more sections

Key Result

Theorem 1

Let $i \in \llbracket nt\rrbracket$, and let $B \in \mathbb{R}^{t \times t}$ such that $BA = \mathop{\mathrm{rref}}\nolimits(A)$. Then $\theta_i$ has a solution in $A$ if and only if there exists $r \in \llbracket t\rrbracket$ such that $B_r$ solves $\theta_i$ in $A$.

Figures (7)

  • Figure 1: A network with 6 users $V$. The adversaries $C = \{ V_2, V_4, V_5 \}$ are shaded. Removing edge $(V_2, V_3)$ would violate our requirements, as adversary $V_2$ would have exactly one non-adversary neighbour.
  • Figure 2: Example graph $G$ with adversaries $C = \{ C_1, C_2, C_3 \}$ (shaded) and non-adversaries $N = N_G(C) = \{ N_1, N_2, N_3 \}$.
  • Figure 3: A graph $G$. Adversaries $C = \{ V_1, V_2, V_3 \}$ are shaded. The bipartite subgraph $H = G[C]$ consists of exactly the non-dotted nodes and edges.
  • Figure 4: Proportion of neighbours' private data that can be reconstructed by adversaries. Each point represents the mean over 1000 random bipartite graphs. Black points indicate no valid bipartite graphs could be found. Note the different y-axes.
  • Figure 7: Example transformation of graph and adversarial knowledge as seen in the proof of \ref{['thm:resistance-by-girth:acyclic-graphs:acyclic-is-no-solution']}.
  • ...and 2 more figures

Theorems & Definitions (14)

  • Definition 1: Adversarial knowledge
  • Remark 1
  • Definition 2: Solution of a variable
  • Remark 2
  • Definition 3: Partial solution
  • Theorem 1
  • Theorem 2
  • Remark 3
  • Lemma 1
  • Corollary 1
  • ...and 4 more