Robust Graph Fine-Tuning with Adversarial Graph Prompting

TL;DR

This work tackles the vulnerability of parameter-efficient fine-tuning (PEFT) for graph neural networks to topology and node feature noise. It introduces Adversarial Graph Prompting (AGP), a bi-level min-max framework that uses an inner joint PGD-based attack to synthesize adversarial noises and an outer prompt-learning step to counteract them, with a total objective $L_{total} = L_{adv} + γ L_{ori} + η L_{consis}$. The authors provide theoretical analysis showing that optimal prompts can absorb both feature and topology perturbations and validate AGP across seven molecular benchmarks under node, topology, and hybrid attacks, achieving superior robustness and competitive clean accuracy compared to baselines. The approach is parameter-efficient, requiring only a small fraction of tunable parameters, and demonstrates universality across different pre-training strategies, offering a practical path to robust graph fine-tuning in real-world applications.

Abstract

Parameter-Efficient Fine-Tuning (PEFT) method has emerged as a dominant paradigm for adapting pre-trained GNN models to downstream tasks. However, existing PEFT methods usually exhibit significant vulnerability to various noise and attacks on graph topology and node attributes/features. To address this issue, for the first time, we propose integrating adversarial learning into graph prompting and develop a novel Adversarial Graph Prompting (AGP) framework to achieve robust graph fine-tuning. Our AGP has two key aspects. First, we propose the general problem formulation of AGP as a min-max optimization problem and develop an alternating optimization scheme to solve it. For inner maximization, we propose Joint Projected Gradient Descent (JointPGD) algorithm to generate strong adversarial noise. For outer minimization, we employ a simple yet effective module to learn the optimal node prompts to counteract the adversarial noise. Second, we demonstrate that the proposed AGP can theoretically address both graph topology and node noise. This confirms the versatility and robustness of our AGP fine-tuning method across various graph noise. Note that, the proposed AGP is a general method that can be integrated with various pre-trained GNN models to enhance their robustness on the downstream tasks. Extensive experiments on multiple benchmark tasks validate the robustness and effectiveness of AGP method compared to state-of-the-art methods.

Figures (7)

• Figure 1: Comparison of ROC-AUC degradation under different adversarial attack targets (node, topology and hybrid) on BACE and TOX21 datasets. 'ROC-AUC Drop' indicates the performance gap between clean and noisy baselines. Existing PEFT methods (GPF GPF, LoRA hulora and AdapterGNN AdapterGNN) show large drops across all attack types, while the proposed AGP exhibits consistently minimal degradation, demonstrating strong robustness against different types of adversarial noise.
• Figure 2: Overall architecture of the proposed Adversarial Graph Prompt (AGP) framework. The finetuning process involves two objectives: (1) Maximization (blue dashed line): maximizing the adversarial loss $\mathcal{L}_{adv}$ by generating more challenging adversarial noises $\mathbf{E}^*_x$ and $\mathbf{E}^*_a$ via JointPGD algorithm. (2) Minimization (red dashed line): minimizing the adversarial loss $\mathcal{L}_{adv}$, original loss $\mathcal{L}_{ori}$ and consistency loss $\mathcal{L}_{consis}$ to tune the prompt module and classifier while keeping the GNN backbone frozen.
• Figure 3: Illustrative examples demonstrating the capability of the proposed graph prompt in mitigating node and topology noise during neighborhood aggregation.
• Figure 4: Robustness evaluation of fine-tuning strategies across varying pre-training models on BACE and TOX21 datasets. The legend displays the average results under various pre-training strategies.
• Figure 5: Comparison of GPU memory usages and inference latency across different fine-tuning methods.

Theorems & Definitions (2)

• Theorem 1
• proof