Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DPAR: Decoupled Graph Neural Networks with Node-Level Differential Privacy

Qiuchen Zhang, Hong kyu Lee, Jing Ma, Jian Lou, Carl Yang, Li Xiong

TL;DR

This paper proposes a D ecoupled GNN with Differentially Differentiated PageRank (DPAR) for training GNNs with an enhanced privacy-utility tradeoff and empirically demonstrates better utilities of DPAR with the same level of node DP compared with state-of-the-art methods.

Abstract

Graph Neural Networks (GNNs) have achieved great success in learning with graph-structured data. Privacy concerns have also been raised for the trained models which could expose the sensitive information of graphs including both node features and the structure information. In this paper, we aim to achieve node-level differential privacy (DP) for training GNNs so that a node and its edges are protected. Node DP is inherently difficult for GNNs because all direct and multi-hop neighbors participate in the calculation of gradients for each node via layer-wise message passing and there is no bound on how many direct and multi-hop neighbors a node can have, so existing DP methods will result in high privacy cost or poor utility due to high node sensitivity. We propose a Decoupled GNN with Differentially Private Approximate Personalized PageRank (DPAR) for training GNNs with an enhanced privacy-utility tradeoff. The key idea is to decouple the feature projection and message passing via a DP PageRank algorithm which learns the structure information and uses the top-$K$ neighbors determined by the PageRank for feature aggregation. By capturing the most important neighbors for each node and avoiding the layer-wise message passing, it bounds the node sensitivity and achieves improved privacy-utility tradeoff compared to layer-wise perturbation based methods. We theoretically analyze the node DP guarantee for the two processes combined together and empirically demonstrate better utilities of DPAR with the same level of node DP compared with state-of-the-art methods.

DPAR: Decoupled Graph Neural Networks with Node-Level Differential Privacy

TL;DR

This paper proposes a D ecoupled GNN with Differentially Differentiated PageRank (DPAR) for training GNNs with an enhanced privacy-utility tradeoff and empirically demonstrates better utilities of DPAR with the same level of node DP compared with state-of-the-art methods.

Abstract

Graph Neural Networks (GNNs) have achieved great success in learning with graph-structured data. Privacy concerns have also been raised for the trained models which could expose the sensitive information of graphs including both node features and the structure information. In this paper, we aim to achieve node-level differential privacy (DP) for training GNNs so that a node and its edges are protected. Node DP is inherently difficult for GNNs because all direct and multi-hop neighbors participate in the calculation of gradients for each node via layer-wise message passing and there is no bound on how many direct and multi-hop neighbors a node can have, so existing DP methods will result in high privacy cost or poor utility due to high node sensitivity. We propose a Decoupled GNN with Differentially Private Approximate Personalized PageRank (DPAR) for training GNNs with an enhanced privacy-utility tradeoff. The key idea is to decouple the feature projection and message passing via a DP PageRank algorithm which learns the structure information and uses the top- neighbors determined by the PageRank for feature aggregation. By capturing the most important neighbors for each node and avoiding the layer-wise message passing, it bounds the node sensitivity and achieves improved privacy-utility tradeoff compared to layer-wise perturbation based methods. We theoretically analyze the node DP guarantee for the two processes combined together and empirically demonstrate better utilities of DPAR with the same level of node DP compared with state-of-the-art methods.
Paper Structure (22 sections, 4 theorems, 3 equations, 12 figures, 2 tables, 3 algorithms)

This paper contains 22 sections, 4 theorems, 3 equations, 12 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. GNNs with Personalized PageRank
  4. Differential Privacy (DP)
  5. DP-SGD and Challenges
  6. DPAR
  7. Differentially Private APPR
  8. Differentially Private GNNs
  9. Experimental Results
  10. Privacy vs. Accuracy Trade-off
  11. Privacy Protection Effectiveness
  12. Effects of Privacy Parameters
  13. Related Work
  14. Conclusion
  15. Proof for Theorem \ref{['theo.privacy_em']}
  16. ...and 7 more sections

Key Result

Corollary 1

durfee2019practical$\mathcal{M}_{\text{Gumbel}}^{k}(u)$ adds the one-shot $\textit{Gumbel}(\Delta(u) / \epsilon)$ noise to each utility score $u(x, r)$ and outputs the k indices with the largest noisy values. For any $\delta \geq 0$, $\mathcal{M}_{\text{Gumbel}}^{k}(u)$ is $\left(\varepsilon^{\prime

Figures (12)

  • Figure 1: Relationship between privacy budget $\epsilon$ (fixed $\delta=2 \times 10^{-3}$) and test accuracy on Cora-ML dataset.
  • Figure 2: Cora-ML. The privacy budget $\epsilon$ ratio for DP-ARRP
  • Figure 3: CS. The privacy budget $\epsilon$ ratio for DP-ARRP
  • Figure 4: MS Academic. The privacy budget $\epsilon$ ratio for DP-ARRP
  • Figure 5: Reddit. The privacy budget $\epsilon$ ratio for DP-ARRP
  • ...and 7 more figures

Theorems & Definitions (6)

  • Definition 1
  • Definition 2
  • Corollary 1
  • Theorem 1
  • Theorem 2
  • Theorem 3