Degree-Preserving Randomized Response for Graph Neural Networks under Local Differential Privacy

TL;DR

The paper tackles edge privacy in unattributed graphs for graph neural networks by introducing Degree-Preserving Randomized Response (DPRR), a one-round local DP method that preserves each user’s degree while providing $\varepsilon$-edge LDP through Laplacian-noised degree estimates and adaptive Warner's RR with degree-aware edge sampling. Key contributions include a two-budget DP framework $(\varepsilon_1, \varepsilon_2)$, a privacy-budget allocation strategy that minimizes total noise, and a formal degree-preservation analysis showing $\mathbb{E}[\tilde{d}_i]\approx d_i$ with bounded variance. Empirical results on three social graph datasets demonstrate that DPRR outperforms baselines (RR, LocalLap, NonPriv-Part) and achieves accuracy close to a fully non-private setting at $\varepsilon\approx 1$, while offering substantial improvements in time and memory efficiency over RR. The work also includes a study of data poisoning attacks and defenses, indicating practical resilience against certain attack classes and outlining directions for stronger defenses and combined edge-feature LDP in future work.

Abstract

Differentially private GNNs (Graph Neural Networks) have been recently studied to provide high accuracy in various tasks on graph data while strongly protecting user privacy. In particular, a recent study proposes an algorithm to protect each user's feature vector in an attributed graph, which includes feature vectors along with node IDs and edges, with LDP (Local Differential Privacy), a strong privacy notion without a trusted third party. However, this algorithm does not protect edges (friendships) in a social graph, hence cannot protect user privacy in unattributed graphs, which include only node IDs and edges. How to provide strong privacy with high accuracy in unattributed graphs remains open. In this paper, we propose a novel LDP algorithm called the DPRR (Degree-Preserving Randomized Response) to provide LDP for edges in GNNs. Our DPRR preserves each user's degree hence a graph structure while providing edge LDP. Technically, our DPRR uses Warner's RR (Randomized Response) and strategic edge sampling, where each user's sampling probability is automatically tuned using the Laplacian mechanism to preserve the degree information under edge LDP. We also propose a privacy budget allocation method to make the noise in both Warner's RR and the Laplacian mechanism small. We focus on graph classification as a task of GNNs and evaluate the DPRR using three social graph datasets. Our experimental results show that the DPRR significantly outperforms three baselines and provides accuracy close to a non-private algorithm in all datasets with a reasonable privacy budget, e.g., epsilon=1. Finally, we introduce data poisoning attacks to our DPRR and a defense against the attacks. We evaluate them using the three social graph datasets and discuss the experimental results.

Figures (9)

• Figure 1: System model. For each graph $G$, users $v_1, \cdots, v_n$ send their noisy neighbor lists $\tilde{\mathbf{a}}_1, \cdots, \tilde{\mathbf{a}}_n$ providing edge LDP. Then, the server calculates a noisy adjacency matrix $\tilde{\mathbf{A}}$ corresponding to $\tilde{\mathbf{a}}_1, \cdots, \tilde{\mathbf{a}}_n$. The server trains the GNN using matrices $\tilde{\mathbf{A}}$ of labeled graphs and predicts a label for each unlabeled graph using its matrix $\tilde{\mathbf{A}}$ and the trained GNN.
• Figure 2: Overview of the DPRR (Degree-Preserving Randomized Response).
• Figure 4: Classification accuracy and AUC for different proportions $\lambda$ of non-private users ($\varepsilon=1$). An error bar represents the standard deviation. Note that the RR is also inefficient in terms of the training/classification time and the memory size.
• Figure 5: Classification accuracy and AUC for different privacy budgets $\varepsilon$ ($\textcolor{black}{\lambda}=0$). An error bar represents the standard deviation. Note that the RR is also inefficient in terms of the training/classification time and the memory size.
• Figure 6: Classification accuracy for different parameters $\alpha$ in our privacy budget allocation method ($\varepsilon=0.2$, $\lambda=0$ or $0.5$). An error bar represents the standard deviation.

Theorems & Definitions (7)

• Definition 1: $\varepsilon$-edge LDP Qin_CCS17
• Definition 2: $\varepsilon$-relationship DP Imola_USENIX21
• Proposition 3: Edge LDP and relationship DP Imola_USENIX21
• Definition 4
• Proposition 5
• Proposition 6
• Lemma 7: Sequential composition of edge LDP Imola_USENIX22