Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lossless Privacy-Preserving Aggregation for Decentralized Federated Learning

Xiaoye Miao, Bin Li, Yanzhang, Xinkui Zhao, Yangyang Wu

TL;DR

LPPA tackles the privacy leakage risk in decentralized federated learning by injecting the noise difference into the initial gradient-tracking variable and leveraging noise flow conservation to globally cancel noise during aggregation. This approach achieves lossless global model accuracy while providing stronger gradient protection, with a formal privacy-budget advantage of a factor of $\\sqrt{2}$ over standard differential privacy. Empirical results on six real datasets demonstrate that LPPA matches or exceeds the accuracy of standard DSGT, while yielding substantially higher resistance to data-reconstruction attacks compared to DP methods. The combination of a preemptive noise-exchange phase and noise-difference injection enables robust privacy without compromising convergence or performance, making it a practical option for privacy-aware DFL in edge and distributed settings.

Abstract

Privacy concerns arise as sensitive data proliferate. Despite decentralized federated learning (DFL) aggregating gradients from neighbors to avoid direct data transmission, it still poses indirect data leaks from the transmitted gradients. Existing privacy-preserving methods for DFL add noise to gradients. They either diminish the model predictive accuracy or suffer from ineffective gradient protection. In this paper, we propose a novel lossless privacy-preserving aggregation rule named LPPA to enhance gradient protection as much as possible but without loss of DFL model predictive accuracy. LPPA subtly injects the noise difference between the sent and received noise into transmitted gradients for gradient protection. The noise difference incorporates neighbors' randomness for each client, effectively safeguarding against data leaks. LPPA employs the noise flow conservation theory to ensure that the noise impact can be globally eliminated. The global sum of all noise differences remains zero, ensuring that accurate gradient aggregation is unaffected and the model accuracy remains intact. We theoretically prove that the privacy-preserving capacity of LPPA is \sqrt{2} times greater than that of noise addition, while maintaining comparable model accuracy to the standard DFL aggregation without noise injection. Experimental results verify the theoretical findings and show that LPPA achieves a 14% mean improvement in accuracy over noise addition. We also demonstrate the effectiveness of LPPA in protecting raw data and guaranteeing lossless model accuracy.

Lossless Privacy-Preserving Aggregation for Decentralized Federated Learning

TL;DR

LPPA tackles the privacy leakage risk in decentralized federated learning by injecting the noise difference into the initial gradient-tracking variable and leveraging noise flow conservation to globally cancel noise during aggregation. This approach achieves lossless global model accuracy while providing stronger gradient protection, with a formal privacy-budget advantage of a factor of over standard differential privacy. Empirical results on six real datasets demonstrate that LPPA matches or exceeds the accuracy of standard DSGT, while yielding substantially higher resistance to data-reconstruction attacks compared to DP methods. The combination of a preemptive noise-exchange phase and noise-difference injection enables robust privacy without compromising convergence or performance, making it a practical option for privacy-aware DFL in edge and distributed settings.

Abstract

Privacy concerns arise as sensitive data proliferate. Despite decentralized federated learning (DFL) aggregating gradients from neighbors to avoid direct data transmission, it still poses indirect data leaks from the transmitted gradients. Existing privacy-preserving methods for DFL add noise to gradients. They either diminish the model predictive accuracy or suffer from ineffective gradient protection. In this paper, we propose a novel lossless privacy-preserving aggregation rule named LPPA to enhance gradient protection as much as possible but without loss of DFL model predictive accuracy. LPPA subtly injects the noise difference between the sent and received noise into transmitted gradients for gradient protection. The noise difference incorporates neighbors' randomness for each client, effectively safeguarding against data leaks. LPPA employs the noise flow conservation theory to ensure that the noise impact can be globally eliminated. The global sum of all noise differences remains zero, ensuring that accurate gradient aggregation is unaffected and the model accuracy remains intact. We theoretically prove that the privacy-preserving capacity of LPPA is \sqrt{2} times greater than that of noise addition, while maintaining comparable model accuracy to the standard DFL aggregation without noise injection. Experimental results verify the theoretical findings and show that LPPA achieves a 14% mean improvement in accuracy over noise addition. We also demonstrate the effectiveness of LPPA in protecting raw data and guaranteeing lossless model accuracy.
Paper Structure (16 sections, 3 theorems, 16 equations, 6 figures, 5 tables, 1 algorithm)

This paper contains 16 sections, 3 theorems, 16 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Solution Overview
  5. Performance Analysis
  6. Lossless Global Model Accuracy
  7. Effective Local Gradient Protection
  8. Experimental Evaluation
  9. Accuracy Comparison
  10. Privacy-Preserving Capacity Comparison
  11. Parameter Evaluation
  12. Data Partitioning Evaluation
  13. Conclusion
  14. Summary of Notations
  15. Proof of Theorem \ref{['lossless']}
  16. ...and 1 more sections

Key Result

Theorem 1

Given the DFL connected topology, all injected noise differences can be globally eliminated, then the accuracy of global model aggregated by LPPA can be equivalent to that of standard DSGT without noise injection. Namely, LPPA attains lossless accuracy.

Figures (6)

  • Figure 1: The procedure illustration of LPPA (from the behaviors of client $c_1$), "WG" represents the model weight and gradient tracking variables for each client. Straight lines indicate information transmission and looped lines illustrate local aggregation.
  • Figure 2: The insight of the LPPA solution from both local and global perspectives.
  • Figure 3: The performance vs. communication round.
  • Figure 4: The reconstructed data under DLG attack.
  • Figure 5: The performance vs. noise scale parameter.
  • ...and 1 more figures

Theorems & Definitions (6)

  • Definition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • proof
  • proof