Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompt-based Unifying Inference Attack on Graph Neural Networks

Yuecen Wei, Xingcheng Fu, Lingyun Liu, Qingyun Sun, Hao Peng, Chunming Hu

TL;DR

The paper addresses privacy risks in graph neural networks by introducing ProIA, a prompt-based unifying inference attack framework. ProIA preserves graph topology during pre-training, generates attack-prompts from the pre-trained model, and employs a disentanglement mechanism to adapt to both attribute and membership inference tasks. Through local/global subgraph information, information bottleneck constraints, and contrastive learning, ProIA yields stronger IA performance and robustness against defenses across several real-world datasets. This framework advances privacy-attacking capabilities on graphs and provides guidance on potential vulnerabilities in graph pre-training pipelines with practical implications for security and privacy in GNN deployments.

Abstract

Graph neural networks (GNNs) provide important prospective insights in applications such as social behavior analysis and financial risk analysis based on their powerful learning capabilities on graph data. Nevertheless, GNNs' predictive performance relies on the quality of task-specific node labels, so it is common practice to improve the model's generalization ability in the downstream execution of decision-making tasks through pre-training. Graph prompting is a prudent choice but risky without taking measures to prevent data leakage. In other words, in high-risk decision scenarios, prompt learning can infer private information by accessing model parameters trained on private data (publishing model parameters in pre-training, i.e., without directly leaking the raw data, is a tacitly accepted trend). However, myriad graph inference attacks necessitate tailored module design and processing to enhance inference capabilities due to variations in supervision signals. In this paper, we propose a novel Prompt-based unifying Inference Attack framework on GNNs, named ProIA. Specifically, ProIA retains the crucial topological information of the graph during pre-training, enhancing the background knowledge of the inference attack model. It then utilizes a unified prompt and introduces additional disentanglement factors in downstream attacks to adapt to task-relevant knowledge. Finally, extensive experiments show that ProIA enhances attack capabilities and demonstrates remarkable adaptability to various inference attacks.

Prompt-based Unifying Inference Attack on Graph Neural Networks

TL;DR

The paper addresses privacy risks in graph neural networks by introducing ProIA, a prompt-based unifying inference attack framework. ProIA preserves graph topology during pre-training, generates attack-prompts from the pre-trained model, and employs a disentanglement mechanism to adapt to both attribute and membership inference tasks. Through local/global subgraph information, information bottleneck constraints, and contrastive learning, ProIA yields stronger IA performance and robustness against defenses across several real-world datasets. This framework advances privacy-attacking capabilities on graphs and provides guidance on potential vulnerabilities in graph pre-training pipelines with practical implications for security and privacy in GNN deployments.

Abstract

Graph neural networks (GNNs) provide important prospective insights in applications such as social behavior analysis and financial risk analysis based on their powerful learning capabilities on graph data. Nevertheless, GNNs' predictive performance relies on the quality of task-specific node labels, so it is common practice to improve the model's generalization ability in the downstream execution of decision-making tasks through pre-training. Graph prompting is a prudent choice but risky without taking measures to prevent data leakage. In other words, in high-risk decision scenarios, prompt learning can infer private information by accessing model parameters trained on private data (publishing model parameters in pre-training, i.e., without directly leaking the raw data, is a tacitly accepted trend). However, myriad graph inference attacks necessitate tailored module design and processing to enhance inference capabilities due to variations in supervision signals. In this paper, we propose a novel Prompt-based unifying Inference Attack framework on GNNs, named ProIA. Specifically, ProIA retains the crucial topological information of the graph during pre-training, enhancing the background knowledge of the inference attack model. It then utilizes a unified prompt and introduces additional disentanglement factors in downstream attacks to adapt to task-relevant knowledge. Finally, extensive experiments show that ProIA enhances attack capabilities and demonstrates remarkable adaptability to various inference attacks.

Paper Structure

This paper contains 26 sections, 6 theorems, 23 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Inference Attack
  4. Prompt Attack on Language Models
  5. Preliminary
  6. Notations
  7. Inference Attack
  8. Graph Prompting
  9. Prompt-based Inference Attack
  10. Overview of ProIA
  11. Pre-training for leaking information
  12. Attack Data Generation
  13. Prompting for adapting Inference Attack with disentanglement
  14. Experiment
  15. Experimental settings
  16. ...and 11 more sections

Key Result

Proposition 1

(The lower bound of $I(Z_X;Y^C)$). For any variational distributions $\mathbb{Q}_{1}\left(\mathbf{Y} ^C \mid \mathbf{Z} _{\mathbf{X} }^{L}\right)$ and $\mathbb{Q}_{2}(\mathbf{Y} ^C)$:

Figures (7)

  • Figure 1: The Venn diagram depicts the respective optimization targets for the tasks with the intersections. $I (C; M)$: increases training-test data disparity, $I (C; A)$: improving data fitting, and $I (C; M; A)$: weakening model generalization and increasing model sensitivity to specific attributes.
  • Figure 2: The overall framework of ProIA.
  • Figure 3: Attack AUC-ROC scores of GCN, GAT, and SAGE (from left to right) against defended models for MIA.
  • Figure 4: Case study.
  • Figure 5: Hyperparameter analysis.
  • ...and 2 more figures

Theorems & Definitions (6)

  • Proposition 1
  • Proposition 2
  • Proposition 1
  • Lemma 1
  • Proposition 2
  • Lemma 2