Collaborative Inference over Wireless Channels with Feature Differential Privacy

TL;DR

This work proposes a novel privacy-preserving collaborative inference mechanism, wherein each edge device in the network secures the privacy of extracted features before transmitting them to a central server for inference.

Abstract

Collaborative inference among multiple wireless edge devices has the potential to significantly enhance Artificial Intelligence (AI) applications, particularly for sensing and computer vision. This approach typically involves a three-stage process: a) data acquisition through sensing, b) feature extraction, and c) feature encoding for transmission. However, transmitting the extracted features poses a significant privacy risk, as sensitive personal data can be exposed during the process. To address this challenge, we propose a novel privacy-preserving collaborative inference mechanism, wherein each edge device in the network secures the privacy of extracted features before transmitting them to a central server for inference. Our approach is designed to achieve two primary objectives: 1) reducing communication overhead and 2) ensuring strict privacy guarantees during feature transmission, while maintaining effective inference performance. Additionally, we introduce an over-the-air pooling scheme specifically designed for classification tasks, which provides formal guarantees on the privacy of transmitted features and establishes a lower bound on classification accuracy.

Figures (11)

• Figure 1: Illustration of the private task-inference framework: Each edge device extracts features from the observed input that preserves some relevant information for classification while satisfies rigorous feature DP levels. Then, each device forwards the processed features over a communication channel to be processed by the central inference server.
• Figure 2: Impact of the dimensionality reduction on the classification accuracy for the same privacy leakage, where $r = q \times 7 \times 7$.
• Figure 3: Entropy Comparison Based on the ModelNet Dataset: The top plot illustrates the Shannon entropy values for each device index, while the bottom plot presents the corresponding Min entropy values. Shannon entropy computes the average uncertainty across possible outcomes, whereas Min entropy is a more restrictive measure, focusing on the most probable outcome. Despite their different calculations, both functions exhibit similar behavior across the devices in the ModelNet dataset, suggesting that the choice between the two measures does not significantly alter the device participation strategy.
• Figure 4: Participation Probability vs. Device Index for Different Thresholds ($\eta$) for Scheme 2.1: This figure shows the participation probabilities $p_k$ for each device index based on three thresholds: $\eta = 0.05$, $\eta = 0.25$, and $\eta = 0.8$. The probabilities $p_k$ are obtained from eqn. \ref{['eq:inputsignal_scheme_2']}, with $u_k$ representing the Shannon entropy for each device and $\sigma_k^{(0)} = \sqrt{0.1}$. The plots illustrate how varying $\eta$ affects the likelihood of device participation, with consistent axis limits across subplots for easier comparison.
• Figure 5: Illustration of the private feature aware transmission for Scheme 2.1. Edge device $k$ participates in the collaborative inference iff the privatized uncertainty score $\tilde{u}_{k}$ is below a certain threshold $\eta$.

Theorems & Definitions (22)

• Definition 1: $(\epsilon, \delta)$-feature DP
• Definition 2
• Remark 1
• Theorem 1
• Remark 2
• Remark 3
• Remark 4
• Definition 3: Classification Margin sokolic2017robust
• Theorem 2: Classification Accuracy
• Lemma 1