Randomized Message-Interception Smoothing: Gray-box Certificates for Graph Neural Networks

TL;DR

Novel gray-box certificates that exploit the message-passing principle of GNNs are proposed, which provide stronger guarantees for attacks at larger distances, as messages from farther-away nodes are more likely to get intercepted.

Abstract

Randomized smoothing is one of the most promising frameworks for certifying the adversarial robustness of machine learning models, including Graph Neural Networks (GNNs). Yet, existing randomized smoothing certificates for GNNs are overly pessimistic since they treat the model as a black box, ignoring the underlying architecture. To remedy this, we propose novel gray-box certificates that exploit the message-passing principle of GNNs: We randomly intercept messages and carefully analyze the probability that messages from adversarially controlled nodes reach their target nodes. Compared to existing certificates, we certify robustness to much stronger adversaries that control entire nodes in the graph and can arbitrarily manipulate node features. Our certificates provide stronger guarantees for attacks at larger distances, as messages from farther-away nodes are more likely to get intercepted. We demonstrate the effectiveness of our method on various models and datasets. Since our gray-box certificates consider the underlying graph structure, we can significantly improve certifiable robustness by applying graph sparsification.

Figures (22)

• Figure 1: Randomized message-interception smoothing: We model adversaries that can arbitrarily manipulate features of multiple nodes in their control (red) to alter the predictions for a target node $v$. We intercept messages (gray) by randomly deleting edges and/or ablating (mask) all features of entire nodes. Our certificates are based on the majority vote under this randomized message interception.
• Figure 2: Single source bounding constant $\Delta_i$ for different edge deletion probabilities $p_{d}$ and node feature ablation probabilities $p_{a}$. White isolines indicate $\Delta_i=0.5$ and separate the theoretically certifiable region ($\Delta_i < 0.5$) from the uncertifiable region ($\Delta_i \geq 0.5$). (a) For the target node, $p_{d}$ does not affect $\Delta_i$. (b) Direct neighbor of target node, single edge. (c) Second-hop neighbor, single path (two edges). (a-c) More distant nodes have larger theoretically certifiable regions.
• Figure 3: Smoothed GAT on Cora-ML: (a) Robustness at different distances to target nodes ($p_{d}$=$0.31$, $p_{a}$=$0.794$, with skip, ACC=$0.79$). (b) Robustness normalized by receptive field size ("attack surface"). (c) Naïve baseline comparison (base certificate bojchevski2020efficient, $10^5$ samples, $\alpha$=$0.01$).
• Figure 4: (a,b) Sparsification significantly improves certifiable robustness of our gray-box certificates to second-hop attacks since sparsification reduces (a) messages to intercept, and (b) receptive field sizes and thus the "attack surface" (Smoothed GAT, Cora-ML, $p_{d}=0.31$, $p_{a}=0.71$, with skip-connection, ACC $=0.8$). (c) Our certificate with largest certifiable radius of 4 with varying samples for certification (Smoothed GAT, Cora-ML, $p_{d}=0$, $p_{a}=0.85$). Our certificates are more sample efficient than existing smoothing-based certificates for GNNs.
• Figure 5: Second-hop attacks on Cora-ML: (a) Robustness-accuracy tradeoffs for different GNN architectures. (b) Skip-connections yield improved robustness-accuracy tradeoffs for node feature ablation smoothing. (c) Ablating less during training yields better robustness-accuracy tradeoffs (GAT).

Theorems & Definitions (51)

• Proposition 1
• Proposition 2
• Lemma 1
• Theorem 1
• Corollary 1: Multi-class certificate
• Theorem 2: Single Source Multiplicative Bound
• Theorem 3: Generalized multiplicative bound
• Corollary 2
• Proposition 3
• Proposition 1