Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Graph Neural Networks via Unbiased Aggregation

Zhichao Hou, Ruiqi Feng, Tyler Derr, Xiaorui Liu

TL;DR

This work dives into the robustness analysis of representative robust GNNs and provides a unified robust estimation point of view to understand their robustness and limitations, and develops an efficient Quasi-Newton Iterative Reweighted Least Squares algorithm to solve the estimation problem.

Abstract

The adversarial robustness of Graph Neural Networks (GNNs) has been questioned due to the false sense of security uncovered by strong adaptive attacks despite the existence of numerous defenses. In this work, we delve into the robustness analysis of representative robust GNNs and provide a unified robust estimation point of view to understand their robustness and limitations. Our novel analysis of estimation bias motivates the design of a robust and unbiased graph signal estimator. We then develop an efficient Quasi-Newton Iterative Reweighted Least Squares algorithm to solve the estimation problem, which is unfolded as robust unbiased aggregation layers in GNNs with theoretical guarantees. Our comprehensive experiments confirm the strong robustness of our proposed model under various scenarios, and the ablation study provides a deep understanding of its advantages. Our code is available at https://github.com/chris-hzc/RUNG.

Robust Graph Neural Networks via Unbiased Aggregation

TL;DR

This work dives into the robustness analysis of representative robust GNNs and provides a unified robust estimation point of view to understand their robustness and limitations, and develops an efficient Quasi-Newton Iterative Reweighted Least Squares algorithm to solve the estimation problem.

Abstract

The adversarial robustness of Graph Neural Networks (GNNs) has been questioned due to the false sense of security uncovered by strong adaptive attacks despite the existence of numerous defenses. In this work, we delve into the robustness analysis of representative robust GNNs and provide a unified robust estimation point of view to understand their robustness and limitations. Our novel analysis of estimation bias motivates the design of a robust and unbiased graph signal estimator. We then develop an efficient Quasi-Newton Iterative Reweighted Least Squares algorithm to solve the estimation problem, which is unfolded as robust unbiased aggregation layers in GNNs with theoretical guarantees. Our comprehensive experiments confirm the strong robustness of our proposed model under various scenarios, and the ablation study provides a deep understanding of its advantages. Our code is available at https://github.com/chris-hzc/RUNG.
Paper Structure (43 sections, 7 theorems, 32 equations, 14 figures, 13 tables)

This paper contains 43 sections, 7 theorems, 32 equations, 14 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Estimation Bias Analysis of Robust GNNs
  3. Robustness Analysis
  4. A Unified View of Robust Estimation
  5. Bias Analysis and Performance Degradation
  6. Robust GNNs with Unbiased Aggregation
  7. Robust and Unbiased Graph Signal Estimator
  8. Quasi-Newton IRLS
  9. GNN with Robust Unbiased Aggregation
  10. Experiment
  11. Experiment Setting
  12. Adversarial Robustness
  13. Ablation study
  14. Related Work
  15. Conclusion & Limitation
  16. ...and 28 more sections

Key Result

Theorem \ref{theorem:gd_descent}

If ${\bm{F}}^{(k)}$ follows the update rule in Eq. eq:rw_update_f1 where $\rho$ defining ${\bm{W}}$ satisfies that $[b]{{\frac{d\rho(y)}{dy^2}}}$ is non-decreasing $\forall y\in(0,\infty)$, then a sufficient condition for $\mathcal{H}({\bm{F}}^{(k+1)})\le \mathcal{H}({\bm{F}}^{(k)})$ is that the ste

Figures (14)

  • Figure 1: Robustness analysis under adaptive local attack. The perturbation budget ($x$-axis) is the number of edges allowed to be perturbed relative to the target node's degree. SoftMedian, TWIRLS, and ElasticGNN (blue curves) exhibit similarly aligned competitive robustness among all the selected robust GNNs, but all models experience catastrophic performance degradation as the attack budget increases.
  • Figure 2: Different mean estimators in the presence of outliers. The clean samples are the majority of data points following the Gaussian distribution $\mathcal{N}((0,0), 1 \cdot I)$, while the outliers are data points that deviate significantly from the main data pattern, following $\mathcal{N}((8,8), 0.5 \cdot I)$. $\ell_2$-estimator deviates far from the true mean, while the $\ell_1$-based estimator is more resistant to outliers. However, as the ratio of outliers escalates, the $\ell_1$-based estimator encounters a greater shift from the true mean, but our estimator still maintains a position close to the ground truth.
  • Figure 3: Penalties.
  • Figure 4: $\frac{d\rho(y)}{d y^2}$.
  • Figure 5: Convergence of our QN-IRLS compared to IRLS.
  • ...and 9 more figures

Theorems & Definitions (14)

  • Theorem \ref{theorem:gd_descent}
  • Theorem \ref{theorem:preconditioned_descent}
  • Lemma 1
  • proof
  • Remark 1
  • Lemma 2
  • proof
  • Remark 2
  • Theorem \ref{theorem:preconditioned_descent}
  • proof
  • ...and 4 more