Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Edge Private Graph Neural Networks with Singular Value Perturbation

Tingting Tang, Yue Niu, Salman Avestimehr, Murali Annavaram

TL;DR

Eclipse introduces a novel privacy-preserving GNN training method that protects graph edges by operating on a low-rank graph representation and applying differential privacy noise only to the singular values. By anchoring to shared principal bases and perturbing a reduced set of singular values, Eclipse achieves strong edge DP with substantially better utility than prior edge-DP methods, especially under tight privacy budgets (e.g., $\\epsilon<4$). Empirical results across multiple datasets show improved F1 scores and reduced attack AUC against LPA and LINKTELLER, with robust performance in both transductive and inductive settings. The approach preserves core graph structure, requires no architectural changes, and demonstrates practical resilience against common edge-leakage attacks, highlighting its potential for secure deployment of GNNs on sensitive graph data.

Abstract

Graph neural networks (GNNs) play a key role in learning representations from graph-structured data and are demonstrated to be useful in many applications. However, the GNN training pipeline has been shown to be vulnerable to node feature leakage and edge extraction attacks. This paper investigates a scenario where an attacker aims to recover private edge information from a trained GNN model. Previous studies have employed differential privacy (DP) to add noise directly to the adjacency matrix or a compact graph representation. The added perturbations cause the graph structure to be substantially morphed, reducing the model utility. We propose a new privacy-preserving GNN training algorithm, Eclipse, that maintains good model utility while providing strong privacy protection on edges. Eclipse is based on two key observations. First, adjacency matrices in graph structures exhibit low-rank behavior. Thus, Eclipse trains GNNs with a low-rank format of the graph via singular values decomposition (SVD), rather than the original graph. Using the low-rank format, Eclipse preserves the primary graph topology and removes the remaining residual edges. Eclipse adds noise to the low-rank singular values instead of the entire graph, thereby preserving the graph privacy while still maintaining enough of the graph structure to maintain model utility. We theoretically show Eclipse provide formal DP guarantee on edges. Experiments on benchmark graph datasets show that Eclipse achieves significantly better privacy-utility tradeoff compared to existing privacy-preserving GNN training methods. In particular, under strong privacy constraints ($ε$ < 4), Eclipse shows significant gains in the model utility by up to 46%. We further demonstrate that Eclipse also has better resilience against common edge attacks (e.g., LPA), lowering the attack AUC by up to 5% compared to other state-of-the-art baselines.

Edge Private Graph Neural Networks with Singular Value Perturbation

TL;DR

Eclipse introduces a novel privacy-preserving GNN training method that protects graph edges by operating on a low-rank graph representation and applying differential privacy noise only to the singular values. By anchoring to shared principal bases and perturbing a reduced set of singular values, Eclipse achieves strong edge DP with substantially better utility than prior edge-DP methods, especially under tight privacy budgets (e.g., ). Empirical results across multiple datasets show improved F1 scores and reduced attack AUC against LPA and LINKTELLER, with robust performance in both transductive and inductive settings. The approach preserves core graph structure, requires no architectural changes, and demonstrates practical resilience against common edge-leakage attacks, highlighting its potential for secure deployment of GNNs on sensitive graph data.

Abstract

Graph neural networks (GNNs) play a key role in learning representations from graph-structured data and are demonstrated to be useful in many applications. However, the GNN training pipeline has been shown to be vulnerable to node feature leakage and edge extraction attacks. This paper investigates a scenario where an attacker aims to recover private edge information from a trained GNN model. Previous studies have employed differential privacy (DP) to add noise directly to the adjacency matrix or a compact graph representation. The added perturbations cause the graph structure to be substantially morphed, reducing the model utility. We propose a new privacy-preserving GNN training algorithm, Eclipse, that maintains good model utility while providing strong privacy protection on edges. Eclipse is based on two key observations. First, adjacency matrices in graph structures exhibit low-rank behavior. Thus, Eclipse trains GNNs with a low-rank format of the graph via singular values decomposition (SVD), rather than the original graph. Using the low-rank format, Eclipse preserves the primary graph topology and removes the remaining residual edges. Eclipse adds noise to the low-rank singular values instead of the entire graph, thereby preserving the graph privacy while still maintaining enough of the graph structure to maintain model utility. We theoretically show Eclipse provide formal DP guarantee on edges. Experiments on benchmark graph datasets show that Eclipse achieves significantly better privacy-utility tradeoff compared to existing privacy-preserving GNN training methods. In particular, under strong privacy constraints ( < 4), Eclipse shows significant gains in the model utility by up to 46%. We further demonstrate that Eclipse also has better resilience against common edge attacks (e.g., LPA), lowering the attack AUC by up to 5% compared to other state-of-the-art baselines.
Paper Structure (33 sections, 4 theorems, 16 equations, 13 figures, 3 tables, 1 algorithm)

This paper contains 33 sections, 4 theorems, 16 equations, 13 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Graph Neural Networks
  5. Differential Privacy
  6. Edge-Level DP on GNN
  7. Problem Settings and Threat Model
  8. Proposed Method
  9. Graph Decomposition
  10. DP on Singular Values
  11. GCN Training and Testing
  12. Privacy Analysis
  13. Formal DP Analysis
  14. Privacy Protection in Practice
  15. Privacy Protection via Low-Rank Decomposition.
  16. ...and 18 more sections

Key Result

lemma 1

Let $s = \texttt{Diag}(U^T \cdot A \cdot V^T)$ and $s' = \texttt{Diag}( U^T \cdot A' \cdot V^T)$, where $U$ and $V$ are principal bases obtained from SVD. Given $A$ and $A'$ share principal bases, we have $\Delta \leq \sqrt{2}$.

Figures (13)

  • Figure 1: A typical 2-layer GCN. Each GCN layer takes as input node embedding, $H^l$, and performs aggregation $\tilde{A}\cdot H^l$ with normalized adjacency matrix $\tilde{A}$. Then, a weight matrix $W$ is applied to the aggregated embeddings.
  • Figure 2: Cosine similarity of principal basis vectors in the Chameleon dataset.
  • Figure 3: Distribution of singular values (normalized) in typical graph data. Singular values decay quickly, indicating that the graph matrix exhibits a highly low-rank structure.
  • Figure 4: GCN with low-rank graph. Eclipse first applies SVD to obtain singular values of a graph, and perturbs singular values with Gaussian noise. Eclipse then reconstructs a low-rank version of $A$ by only keeping most principal components. To align with original format of the adjacency matrix, Eclipse applies a binary quantization before feeding $A_{\text{lr}}$ to a GCN.
  • Figure 5: Model utility in transductive setting for various $\epsilon$. Eclipse achieves much better model utility under high privacy constraints ($\epsilon<6$) compared to DPGCN and LPGNet. Even under strong privacy constraints with $\epsilon<1$, Eclipse still achieves better performance compared to MLP without using adjacency matrices during training.
  • ...and 8 more figures

Theorems & Definitions (8)

  • Definition 1
  • Definition 2
  • lemma 1
  • Theorem 1
  • Corollary 1
  • Remark 1
  • Remark 2
  • Theorem 1