Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GCON: Differentially Private Graph Convolutional Network via Objective Perturbation

Jianxin Wei, Yizheng Zhu, Xiaokui Xiao, Ergute Bao, Yin Yang, Kuntai Cai, Beng Chin Ooi

TL;DR

This work tackles privacy in graph neural networks by addressing edge-level differential privacy for GCNs. It introduces GCON, a method that trains GCNs under edge-DP via objective perturbation while keeping the graph convolution operation intact, thereby preserving message passing. The approach leverages an SGC-based convex formulation, a feature encoder to reduce dimensionality, and tight sensitivity bounds for PPR/APPR propagation to calibrate noise. The authors provide a formal DP analysis and demonstrate through experiments on four real-world datasets that GCON outperforms existing DP methods across diverse privacy budgets, with robust performance on both homophily and heterophily graphs. This work enables high-utility, privacy-preserving graph learning suitable for sensitive domains like social networks and healthcare analytics.

Abstract

Graph Convolutional Networks (GCNs) are a popular machine learning model with a wide range of applications in graph analytics, including healthcare, transportation, and finance. However, a GCN trained without privacy protection measures may memorize private interpersonal relationships in the training data through its model parameters. This poses a substantial risk of compromising privacy through link attacks, potentially leading to violations of privacy regulations such as GDPR. To defend against such attacks, a promising approach is to train the GCN with differential privacy (DP), a rigorous framework that provides strong privacy protection by injecting random noise into the training process. However, training a GCN under DP is a highly challenging task. Existing solutions either perturb the graph topology or inject randomness into the graph convolution operations, or overestimate the amount of noise required, resulting in severe distortions of the network's message aggregation and, thus, poor model utility. Motivated by this, we propose GCON, a novel and effective solution for training GCNs with edge differential privacy. GCON leverages the classic idea of perturbing the objective function to satisfy DP and maintains an unaltered graph convolution process. Our rigorous theoretical analysis offers tight, closed-form bounds on the sensitivity of the graph convolution results and quantifies the impact of an edge modification on the trained model parameters. Extensive experiments using multiple benchmark datasets across diverse settings demonstrate the consistent superiority of GCON over existing solutions.

GCON: Differentially Private Graph Convolutional Network via Objective Perturbation

TL;DR

This work tackles privacy in graph neural networks by addressing edge-level differential privacy for GCNs. It introduces GCON, a method that trains GCNs under edge-DP via objective perturbation while keeping the graph convolution operation intact, thereby preserving message passing. The approach leverages an SGC-based convex formulation, a feature encoder to reduce dimensionality, and tight sensitivity bounds for PPR/APPR propagation to calibrate noise. The authors provide a formal DP analysis and demonstrate through experiments on four real-world datasets that GCON outperforms existing DP methods across diverse privacy budgets, with robust performance on both homophily and heterophily graphs. This work enables high-utility, privacy-preserving graph learning suitable for sensitive domains like social networks and healthcare analytics.

Abstract

Graph Convolutional Networks (GCNs) are a popular machine learning model with a wide range of applications in graph analytics, including healthcare, transportation, and finance. However, a GCN trained without privacy protection measures may memorize private interpersonal relationships in the training data through its model parameters. This poses a substantial risk of compromising privacy through link attacks, potentially leading to violations of privacy regulations such as GDPR. To defend against such attacks, a promising approach is to train the GCN with differential privacy (DP), a rigorous framework that provides strong privacy protection by injecting random noise into the training process. However, training a GCN under DP is a highly challenging task. Existing solutions either perturb the graph topology or inject randomness into the graph convolution operations, or overestimate the amount of noise required, resulting in severe distortions of the network's message aggregation and, thus, poor model utility. Motivated by this, we propose GCON, a novel and effective solution for training GCNs with edge differential privacy. GCON leverages the classic idea of perturbing the objective function to satisfy DP and maintains an unaltered graph convolution process. Our rigorous theoretical analysis offers tight, closed-form bounds on the sensitivity of the graph convolution results and quantifies the impact of an edge modification on the trained model parameters. Extensive experiments using multiple benchmark datasets across diverse settings demonstrate the consistent superiority of GCON over existing solutions.
Paper Structure (42 sections, 13 theorems, 98 equations, 4 figures, 2 tables, 4 algorithms)

This paper contains 42 sections, 13 theorems, 98 equations, 4 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Convolutional Networks
  4. SGC and PPNP
  5. Differential Privacy
  6. Problem Setting
  7. GCON
  8. High-Level Idea and Major Challenges
  9. Solution Overview
  10. Detailed GCON Model
  11. Feature Encoder
  12. Propagation with PPR and APPR
  13. SGC-Based Network Architecture
  14. Strongly-Convex Loss Function
  15. Objective Perturbation
  16. ...and 27 more sections

Key Result

Theorem 1

Algorithm alg:Train satisfies ($\epsilon,\delta$)-DP, with $\Lambda'$ and $\beta$ set to be Eq. (eq:Lambda_prime) - (eq:beta), where $c, d, \Lambda, \ell(x;y), n_1, \omega, \alpha, \{m_i\}_{i=1}^s$ are inputs and other parameters $c_1,c_2,c_3, \Psi(\bm{Z}), c_{sf}, c_{\theta}, \epsilon_{\Lambda}$ are sequentially computed by Eq. (eq:supremum of l(x;y)) - (eq:epsilon_Lambda).

Figures (4)

  • Figure 1: Model performance (micro F1 score) versus privacy constraints on different datasets. GCN (non-DP) represents the target performance (i.e., an upper bound) for all DP algorithms.
  • Figure 2: Effect of the propagation step in GCON on private test graph under $\epsilon=4$.
  • Figure 3: Effect of the propagation step in GCON on public test graph under $\epsilon=4$.
  • Figure 4: Effect of the restart probability $\alpha$ in GCON.

Theorems & Definitions (31)

  • Definition 1: ($\epsilon, \delta$)-Differential Privacy (DP) dwork2006differential
  • Definition 2: Neighboring datasets
  • Theorem 1
  • Lemma 1
  • Definition 3: Sensitivity metric
  • Lemma 2
  • Definition 4: Convex and strictly convex
  • Definition 5: Strongly convex
  • Lemma 3
  • proof
  • ...and 21 more