On Measuring Unnoticeability of Graph Adversarial Attacks: Observations, New Measure, and Applications

TL;DR

This work tackles the problem of measuring graph attack noticeability by identifying two core deficits in prior metrics: bypassability and overlooking small perturbations. It introduces HideNSeek, a learnable noticeability measure that uses a Learnable Edge Scorer (LEO) to rank edges by attack-likelihood and AUROC-based aggregation to produce a robust final score. Empirical results across six real-world graphs show LEO outperforms a dozen baselines in detecting attack edges and HideNSeek yields significantly lower bypassability and greater sensitivity at small attack rates, while also improving GNN robustness when used to prune attack-like edges. The approach extends to node-feature attacks via LFO and demonstrates practical impact for enhancing graph-based learning systems.

Abstract

Adversarial attacks are allegedly unnoticeable. Prior studies have designed attack noticeability measures on graphs, primarily using statistical tests to compare the topology of original and (possibly) attacked graphs. However, we observe two critical limitations in the existing measures. First, because the measures rely on simple rules, attackers can readily enhance their attacks to bypass them, reducing their attack "noticeability" and, yet, maintaining their attack performance. Second, because the measures naively leverage global statistics, such as degree distributions, they may entirely overlook attacks until severe perturbations occur, letting the attacks be almost "totally unnoticeable." To address the limitations, we introduce HideNSeek, a learnable measure for graph attack noticeability. First, to mitigate the bypass problem, HideNSeek learns to distinguish the original and (potential) attack edges using a learnable edge scorer (LEO), which scores each edge on its likelihood of being an attack. Second, to mitigate the overlooking problem, HideNSeek conducts imbalance-aware aggregation of all the edge scores to obtain the final noticeability score. Using six real-world graphs, we empirically demonstrate that HideNSeek effectively alleviates the observed limitations, and LEO (i.e., our learnable edge scorer) outperforms eleven competitors in distinguishing attack edges under five different attack methods. For an additional application, we show that LEO boost the performance of robust GNNs by removing attack-like edges.

Figures (6)

• Figure 1: The existing noticeability measures are bypassable. The y-axis stands for noticeability (the attack is more noticeable when it is higher) and the x-axis stands for node classification accuracy (the attack is more effective when it is lower). Circle markers ($\medcirc$) represent original attacks generated through PGD Attack on Cora, while star markers ($\medwhitestar$) indicate adaptive attacks aiming to minimize the corresponding noticeability. The color filled in each marker indicates the attack rate. Note that adaptive attacks significantly reduce the noticeability measures while largely maintaining the attack performance.
• Figure 2: The existing noticeability measures are overlooking. The y-axis stands for noticeability scores (the attack is more noticeable when it is higher) and the x-axis stands for the attack rate. Each black dashed line indicates a threshold where the attack is considered noticeable with statistical significance (with $p = 0.05$). Note that the noticeability scores remain near-zero until the attack rate $\gamma$ reaches a sufficiently high level (around 5-12%).
• Figure 3: (Left) Attack phase where some attack edges can be added (red line) and some original edges can be deleted (blue dashed line). (Right) The procedure of computing HideNSeek. Using the edge scores from LEO (see Fig. \ref{['fig:model']} for its details) as (soft) predictions, the noticeability score w.r.t. HideNSeek is computed as the AUROC score. The attack is considered more noticeable if the AUROC is higher.
• Figure 4: Structure of LEO. LEO uses an ensemble model with three modules, a vanilla GNN module ($M_G$), a GSL-based GNN module ($M_S$), and a node proximity module ($M_P$). For each node pair, each module learns or computes a sub-score and an attention value, and the final score of the pair is the sum of its sub-scores weighted by their attention values.
• Figure 5: Further application:LEO consistently improves the node classification accuracy of GCN ((a)-(e)), RGCN ((f)-(j)), and MedianGCN ((k)-(o)). Even for MedianGCN (a robust GNN), LEO yields up to 12.4% better accuracy than the best competitor.

Theorems & Definitions (3)

• Definition 1: Adaptive Attack
• Definition 2: Bypassability
• Definition 3: Overlookingness