Adversarial Training for Graph Neural Networks via Graph Subspace Energy Optimization

TL;DR

This paper tackles adversarial topology perturbations in inductive graph learning by introducing Graph Subspace Energy (GSE), a stability proxy defined as $\mathrm{GSE}(A,\beta_1,\beta_2) = \sum_{i=\lceil \beta_1 n \rceil}^{\lfloor \beta_2 n \rfloor} \sigma_i(A)$ that focuses on a subspace of singular values of the adjacency matrix. It then proposes AT-GSE, a minimax training framework that uses perturbations maximizing GSE (including $\mathrm{GSE}_{\alpha}$ offsets) to forge robust graphs during training, while training GNNs to minimize the augmented loss; two efficient perturbation schemes, RndGSE and Nyström, balance robustness and scalability. Extensive experiments across seven datasets in fully inductive semi-supervised settings show that AT-GSE improves adversarial robustness against LRBCD and PRBCD attacks and also enhances clean accuracy on non-perturbed graphs, with Nyström excelling at defending global perturbations and RndSVD at local perturbations. The proposed approach provides a scalable, effective defense for inductive graph learning and highlights GSE as a practical measure of GNN robustness, with future work aimed at deeper theoretical grounding and scalability to even larger graphs.

Abstract

Despite impressive capability in learning over graph-structured data, graph neural networks (GNN) suffer from adversarial topology perturbation in both training and inference phases. While adversarial training has demonstrated remarkable effectiveness in image classification tasks, its suitability for GNN models has been doubted until a recent advance that shifts the focus from transductive to inductive learning. Still, GNN robustness in the inductive setting is under-explored, and it calls for deeper understanding of GNN adversarial training. To this end, we propose a new concept of graph subspace energy (GSE) -- a generalization of graph energy that measures graph stability -- of the adjacency matrix, as an indicator of GNN robustness against topology perturbations. To further demonstrate the effectiveness of such concept, we propose an adversarial training method with the perturbed graphs generated by maximizing the GSE regularization term, referred to as AT-GSE. To deal with the local and global topology perturbations raised respectively by LRBCD and PRBCD, we employ randomized SVD (RndSVD) and Nystrom low-rank approximation to favor the different aspects of the GSE terms. An extensive set of experiments shows that AT-GSE outperforms consistently the state-of-the-art GNN adversarial training methods over different homophily and heterophily datasets in terms of adversarial accuracy, whilst more surprisingly achieving a superior clean accuracy on non-perturbed graphs.

Figures (6)

• Figure 1: Normalized graph subspace energy (GSE) of different datasets under adversarial topology attacks with different attack ratios.
• Figure 2: A visual example of GSE under topology attacks of LRBCD with different attack ratios on Cora and Citeseer datasets.
• Figure 3: The distribution shift of singular values of graph adjacency matrix under topology attacks of PRBCD and LRBCD with different attack ratios on Cora and Citeseer datasets.
• Figure 4: The accuracy of each model on the Cora dataset under LRBCD (left) or PRBCD (right) attacks. In the legend, the former is the backbone model and the latter is the name of the method for generating adversarial examples. For example, GPRGNN RndSVD is an adversarial training with GPRGNN as the backbone model and RndSVD generating adversarial examples. The red line in the figure is the method we proposed, and the rest are baselines.
• Figure 5: The accuracy of each model on the Citeseer dataset under LRBCD (left) or PRBCD (right) attacks. In the legend, the former is the backbone model and the latter is the name of the method for generating adversarial examples. For example, GPRGNN RndSVD is an adversarial training with GPRGNN as the backbone model and RndSVD generating adversarial examples. The red line in the figure is the method we proposed, and the rest are baselines.

Theorems & Definitions (1)

• Definition 1: Graph Subspace Energy