Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How does Graph Structure Modulate Membership-Inference Risk for Graph Neural Networks?

Megha Khosla

TL;DR

This work addresses node-level membership inference in graph neural networks, showing that graph structure and inference-time edge access are primary drivers of MI risk beyond the conventional generalization gap. It formalizes node-level MI under subgraph sampling and attack models, and evaluates how training-graph construction (random vs snowball) and edge access regimes (Orig, AllEdges, NoGraph) affect membership advantage across Cora, Chameleon, and PubMed for GCN, GraphSage, and GAT. Key findings include that snowball sampling can increase the generalization gap and thus MI risk, inter-train-test edges at inference can reduce MI leakage but depend on model and dataset, and that the generalization gap is not a reliable proxy for MI risk in graph settings. The work further demonstrates that inductive train-test splits violate statistical exchangeability, limiting the applicability of standard DP membership bounds, and advocates graph-aware privacy auditing and tailored DP definitions for graph-structured data.

Abstract

Graph neural networks (GNNs) have become the standard tool for encoding data and their complex relationships into continuous representations, improving prediction accuracy in several machine learning tasks like node classification and link prediction. However, their use in sensitive applications has raised concerns about the potential leakage of training data. Research on privacy leakage in GNNs has largely been shaped by findings from non-graph domains, such as images and tabular data. We emphasize the need of graph specific analysis and investigate the impact of graph structure on node level membership inference. We formalize MI over node-neighbourhood tuples and investigate two important dimensions: (i) training graph construction and (ii) inference-time edge access. Empirically, snowball's coverage bias often harms generalisation relative to random sampling, while enabling inter-train-test edges at inference improves test accuracy, shrinks the train-test gap, and yields the lowest membership advantage across most of the models and datasets. We further show that the generalisation gap empirically measured as the performance difference between the train and test nodes is an incomplete proxy for MI risk: access to edges dominates-MI can rise or fall independent of gap changes. Finally, we examine the auditability of differentially private GNNs, adapting the definition of statistical exchangeability of train-test data points for graph based models. We show that for node level tasks the inductive splits (random or snowball sampled) break exchangeability, limiting the applicability of standard bounds for membership advantage of differential private models.

How does Graph Structure Modulate Membership-Inference Risk for Graph Neural Networks?

TL;DR

This work addresses node-level membership inference in graph neural networks, showing that graph structure and inference-time edge access are primary drivers of MI risk beyond the conventional generalization gap. It formalizes node-level MI under subgraph sampling and attack models, and evaluates how training-graph construction (random vs snowball) and edge access regimes (Orig, AllEdges, NoGraph) affect membership advantage across Cora, Chameleon, and PubMed for GCN, GraphSage, and GAT. Key findings include that snowball sampling can increase the generalization gap and thus MI risk, inter-train-test edges at inference can reduce MI leakage but depend on model and dataset, and that the generalization gap is not a reliable proxy for MI risk in graph settings. The work further demonstrates that inductive train-test splits violate statistical exchangeability, limiting the applicability of standard DP membership bounds, and advocates graph-aware privacy auditing and tailored DP definitions for graph-structured data.

Abstract

Graph neural networks (GNNs) have become the standard tool for encoding data and their complex relationships into continuous representations, improving prediction accuracy in several machine learning tasks like node classification and link prediction. However, their use in sensitive applications has raised concerns about the potential leakage of training data. Research on privacy leakage in GNNs has largely been shaped by findings from non-graph domains, such as images and tabular data. We emphasize the need of graph specific analysis and investigate the impact of graph structure on node level membership inference. We formalize MI over node-neighbourhood tuples and investigate two important dimensions: (i) training graph construction and (ii) inference-time edge access. Empirically, snowball's coverage bias often harms generalisation relative to random sampling, while enabling inter-train-test edges at inference improves test accuracy, shrinks the train-test gap, and yields the lowest membership advantage across most of the models and datasets. We further show that the generalisation gap empirically measured as the performance difference between the train and test nodes is an incomplete proxy for MI risk: access to edges dominates-MI can rise or fall independent of gap changes. Finally, we examine the auditability of differentially private GNNs, adapting the definition of statistical exchangeability of train-test data points for graph based models. We show that for node level tasks the inductive splits (random or snowball sampled) break exchangeability, limiting the applicability of standard bounds for membership advantage of differential private models.
Paper Structure (27 sections, 1 theorem, 22 equations, 7 figures, 14 tables)

This paper contains 27 sections, 1 theorem, 22 equations, 7 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Graph Neural Networks
  4. Aggregation operations in GNNs
  5. Membership inference attacks and defenses
  6. Node-level Membership Inference
  7. Attack Model
  8. Evaluation using Membership Advantage
  9. Datasets and graph sampling strategies
  10. Datasets
  11. Graph Sampling strategies
  12. Settings for generating training splits with snowball and random sampling.
  13. Influence of sampling strategy and edge access
  14. Characteristics of sampled nodes
  15. Node neighborhoods and output distributions
  16. ...and 12 more sections

Key Result

Theorem 1

Let $\mathcal{D}$ be a joint distribution over $n$ member samples $\{z_1, \ldots, z_n\}$ and a non-member sample $z_{n+1}$, where each sample $z_v$ is defined as in Definition def:stats, and where neighborhoods $\mathcal{N}^L_{G'}(v)$ are computed over a graph view $G'$ constructed via a node sampli

Figures (7)

  • Figure 1: Distribution of KL divergence among posterior distribution of nodes on Cora dataset comparing the cases when all edges and none of the edges were used during inference. The model (Gcn) was trained on a snowball sampled split with 50% of the nodes in the train set.
  • Figure 2: Distribution of KL divergence among posterior distribution of nodes on Cora dataset comparing the cases when all edges and none of the edges were used during inference. The model (Gcn) was trained on a randomly sampled split with 50% of the nodes in the train set.
  • Figure 3: Empirical cumulative distribution function of JS divergence for the three models under different access level settings. Dataset used here is Cora with 10% nodes in train split sampled using snowball sampling. The JS divergence is computed between the query's and neighbors posterior distribution where two nodes are considered when an edge is present between them according to the original split, i.e., no edges are present between the train and test nodes. Overall, one would expect ECDF to be lowest when none of the edges are used during inference. GraphSage shows an exception where posterior distribution of neighboring nodes are less distinguishable even when none of the edges are included in the inference stage.
  • Figure 4: Effect of edge structure on train--test separability of class confidence (log-odds of the true-class posterior) for a GraphSage model. The model was trained over Cora dataset with 10% of nodes sampled for training using snowball sampling. We compare three inference graphs: (a) Original (train/test edge sets disjoint), (b) Full graph (all edges available), and (c) No edges. Train (blue) and test (red) distributions are most separated in (a), indicating stronger distinguishability; in (b) and (c) they overlap more, indicating weaker distinguishability.
  • Figure 5: (a) Performance gap and (b) membership advantage on the Cora dataset at different train sizes (10% (l) and 50% (r)).
  • ...and 2 more figures

Theorems & Definitions (4)

  • Definition 1: Joint Distribution $\mathcal{D}$ over Node Tuples
  • Definition 2: Statistical Exchangeability in MI for GNNs
  • Theorem 1
  • proof