Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Post-Hoc Robustness Enhancement in Graph Neural Networks with Conditional Random Fields

Yassine Abbahaddou, Sofiane Ennadir, Johannes F. Lutzeyer, Fragkiskos D. Malliaros, Michalis Vazirgiannis

TL;DR

This study introduces RobustCRF, a post-hoc approach aiming to enhance the robustness of GNNs at the inference stage, founded on statistical relational learning using a Conditional Random Field, is model-agnostic and does not require prior knowledge about the underlying model architecture.

Abstract

Graph Neural Networks (GNNs), which are nowadays the benchmark approach in graph representation learning, have been shown to be vulnerable to adversarial attacks, raising concerns about their real-world applicability. While existing defense techniques primarily concentrate on the training phase of GNNs, involving adjustments to message passing architectures or pre-processing methods, there is a noticeable gap in methods focusing on increasing robustness during inference. In this context, this study introduces RobustCRF, a post-hoc approach aiming to enhance the robustness of GNNs at the inference stage. Our proposed method, founded on statistical relational learning using a Conditional Random Field, is model-agnostic and does not require prior knowledge about the underlying model architecture. We validate the efficacy of this approach across various models, leveraging benchmark node classification datasets.

Post-Hoc Robustness Enhancement in Graph Neural Networks with Conditional Random Fields

TL;DR

This study introduces RobustCRF, a post-hoc approach aiming to enhance the robustness of GNNs at the inference stage, founded on statistical relational learning using a Conditional Random Field, is model-agnostic and does not require prior knowledge about the underlying model architecture.

Abstract

Graph Neural Networks (GNNs), which are nowadays the benchmark approach in graph representation learning, have been shown to be vulnerable to adversarial attacks, raising concerns about their real-world applicability. While existing defense techniques primarily concentrate on the training phase of GNNs, involving adjustments to message passing architectures or pre-processing methods, there is a noticeable gap in methods focusing on increasing robustness during inference. In this context, this study introduces RobustCRF, a post-hoc approach aiming to enhance the robustness of GNNs at the inference stage. Our proposed method, founded on statistical relational learning using a Conditional Random Field, is model-agnostic and does not require prior knowledge about the underlying model architecture. We validate the efficacy of this approach across various models, leveraging benchmark node classification datasets.

Paper Structure

This paper contains 22 sections, 2 theorems, 40 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Graph Neural Networks
  5. Conditioned Random Fields
  6. RobustCRF: A CRF-based Robustness Enhancement Model
  7. Adversarial Attacks
  8. Motivation
  9. Modeling the Robustness Constraint in a CRF
  10. Mean Field Approximation
  11. Reducing the Size of the CRF
  12. Experimental Setup
  13. Experimental Setup
  14. Experimental Results
  15. Conclusion
  16. ...and 7 more sections

Key Result

Lemma 4.1

By solving the system of Eq. eq:new_objective, we can get the optimal distribution $Q^*$ as follows:

Figures (2)

  • Figure 1: Illustration of RobustCRF. We use input graphs manifold to generate the structure of the CRF, i.e., $V^{\text{CRF}}, E^{\text{CRF}}$. We use the GNN's predictions to generate the observables $\left \{ Y_a ~~~| ~~a \in V^{\text{CRF}} \right \}$, we then run the CRF inference to generate the new GNN's predictions $\{ \tilde{Y}_a ~~~| ~~a \in V^{\text{CRF}} \}$.
  • Figure 2: The effect of the radius on the lower bound stated in Lemma \ref{['lem:size_CRF']}.

Theorems & Definitions (5)

  • Lemma 4.1
  • Lemma 4.2
  • Remark 4.3
  • proof
  • proof