Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leakage-Resilient and Carbon-Neutral Aggregation Featuring the Federated AI-enabled Critical Infrastructure

Zehang Deng, Ruoxi Sun, Minhui Xue, Sheng Wen, Seyit Camtepe, Surya Nepal, Yang Xiang

TL;DR

This paper tackles data leakage and communication bottlenecks in federated learning for AI-enabled critical infrastructures by introducing CDPA, a leakage-resilient, communication-efficient, and carbon-neutral aggregation method. CDPA fuses a novel bit-flipping privacy mechanism with Subtractive Dithered Lattice Quantization (SDQ) to provide provable differential privacy while halving communication costs and maintaining model utility in CV and NLP tasks. The authors prove DP guarantees for bit-flipping, establish utility convergence bounds, and demonstrate robust defense against state-of-the-art data reconstruction attacks, outperforming existing gradient compression and privacy methods. The work further shows CDPA’s practicality in reducing carbon emissions and its applicability to large-scale ACIs, suggesting a path toward privacy-utility-energy-efficient federated learning in space, 6G networks, and agriculture.

Abstract

AI-enabled critical infrastructures (ACIs) integrate artificial intelligence (AI) technologies into various essential systems and services that are vital to the functioning of society, offering significant implications for efficiency, security and resilience. While adopting decentralized AI approaches (such as federated learning technology) in ACIs is plausible, private and sensitive data are still susceptible to data reconstruction attacks through gradient optimization. In this work, we propose Compressed Differentially Private Aggregation (CDPA), a leakage-resilient, communication-efficient, and carbon-neutral approach for ACI networks. Specifically, CDPA has introduced a novel random bit-flipping mechanism as its primary innovation. This mechanism first converts gradients into a specific binary representation and then selectively flips masked bits with a certain probability. The proposed bit-flipping introduces a larger variance to the noise while providing differentially private protection and commendable efforts in energy savings while applying vector quantization techniques within the context of federated learning. The experimental evaluation indicates that CDPA can reduce communication cost by half while preserving model utility. Moreover, we demonstrate that CDPA can effectively defend against state-of-the-art data reconstruction attacks in both computer vision and natural language processing tasks. We highlight existing benchmarks that generate 2.6x to over 100x more carbon emissions than CDPA. We hope that the CDPA developed in this paper can inform the federated AI-enabled critical infrastructure of a more balanced trade-off between utility and privacy, resilience protection, as well as a better carbon offset with less communication overhead.

Leakage-Resilient and Carbon-Neutral Aggregation Featuring the Federated AI-enabled Critical Infrastructure

TL;DR

This paper tackles data leakage and communication bottlenecks in federated learning for AI-enabled critical infrastructures by introducing CDPA, a leakage-resilient, communication-efficient, and carbon-neutral aggregation method. CDPA fuses a novel bit-flipping privacy mechanism with Subtractive Dithered Lattice Quantization (SDQ) to provide provable differential privacy while halving communication costs and maintaining model utility in CV and NLP tasks. The authors prove DP guarantees for bit-flipping, establish utility convergence bounds, and demonstrate robust defense against state-of-the-art data reconstruction attacks, outperforming existing gradient compression and privacy methods. The work further shows CDPA’s practicality in reducing carbon emissions and its applicability to large-scale ACIs, suggesting a path toward privacy-utility-energy-efficient federated learning in space, 6G networks, and agriculture.

Abstract

AI-enabled critical infrastructures (ACIs) integrate artificial intelligence (AI) technologies into various essential systems and services that are vital to the functioning of society, offering significant implications for efficiency, security and resilience. While adopting decentralized AI approaches (such as federated learning technology) in ACIs is plausible, private and sensitive data are still susceptible to data reconstruction attacks through gradient optimization. In this work, we propose Compressed Differentially Private Aggregation (CDPA), a leakage-resilient, communication-efficient, and carbon-neutral approach for ACI networks. Specifically, CDPA has introduced a novel random bit-flipping mechanism as its primary innovation. This mechanism first converts gradients into a specific binary representation and then selectively flips masked bits with a certain probability. The proposed bit-flipping introduces a larger variance to the noise while providing differentially private protection and commendable efforts in energy savings while applying vector quantization techniques within the context of federated learning. The experimental evaluation indicates that CDPA can reduce communication cost by half while preserving model utility. Moreover, we demonstrate that CDPA can effectively defend against state-of-the-art data reconstruction attacks in both computer vision and natural language processing tasks. We highlight existing benchmarks that generate 2.6x to over 100x more carbon emissions than CDPA. We hope that the CDPA developed in this paper can inform the federated AI-enabled critical infrastructure of a more balanced trade-off between utility and privacy, resilience protection, as well as a better carbon offset with less communication overhead.
Paper Structure (37 sections, 2 theorems, 20 equations, 17 figures, 8 tables, 1 algorithm)

This paper contains 37 sections, 2 theorems, 20 equations, 17 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Leakage Risks in ACIs
  4. Communication Bottlenecks in ACIs
  5. Threat Model
  6. CDPA Methodology
  7. Step 0: Initialization
  8. Step 1: Gradient Quantization
  9. Step 2: Privacy Enhancement
  10. Step 3: Secure Addition & Recovery
  11. Privacy Guarantee of Bit-Flipping
  12. Utility Guarantee of CDPA
  13. Experiment Setup
  14. Models
  15. Attacks
  16. ...and 22 more sections

Key Result

Theorem 1

With parameter selection $\hat{\beta} := (\rho^2 \beta_2 + \beta \beta_1)$, then the CDPA algorithm enjoys the following utility bound

Figures (17)

  • Figure 1: An example of AI-enabled critical infrastructure. High-resolution data captured by remote sensing satellites in GEO pave the way for AI-driven advancements. Federated learning emerges as the solution of choice, as satellite communication struggles with image transmission and privacy protection.
  • Figure 2: Overview of CDPA. During the client-side gradient quantization process, gradient updates are quantized at a fixed rate. Then, in selected layers, each bit in the binary representation is flipped according to a flipping mask with probability $1-p$. At the server-side, binary data are aggregated and then recovered.
  • Figure 3: An example of gradient encoding (bit-flipping) on the client-side and decoding (aggravation with flipping restoration) on the server-side. While the bit-flipping may affect the output of a particular client, the restoration process aims to restore the gradient. Specifically, the flipping restoration aggregates and restores the masked bits that have been flipped (highlighted in red) according to the flipping probability $1-p$. The accuracy of the decoding process increases with the number of clients participating in the process, denoted by $R$.
  • Figure 4: Relationship between recovery error induced by bit-flipping and probability $p$.
  • Figure 5: A comparison of CDPA with other schemes on model utility.
  • ...and 12 more figures

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2