Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Key Principles of Graph Machine Learning: Representation, Robustness, and Generalization

Yassine Abbahaddou

TL;DR

This dissertation investigates core aspects of GNNs through developing new representation learning techniques based on Graph Shift Operators (GSOs), and introducing generalization-enhancing methods through graph data augmentation, which provides a more principled understanding of the limitations and potential of GNNs.

Abstract

Graph Neural Networks (GNNs) have emerged as powerful tools for learning representations from structured data. Despite their growing popularity and success across various applications, GNNs encounter several challenges that limit their performance. in their generalization, robustness to adversarial perturbations, and the effectiveness of their representation learning capabilities. In this dissertation, I investigate these core aspects through three main contributions: (1) developing new representation learning techniques based on Graph Shift Operators (GSOs, aiming for enhanced performance across various contexts and applications, (2) introducing generalization-enhancing methods through graph data augmentation, and (3) developing more robust GNNs by leveraging orthonormalization techniques and noise-based defenses against adversarial attacks. By addressing these challenges, my work provides a more principled understanding of the limitations and potential of GNNs.

Key Principles of Graph Machine Learning: Representation, Robustness, and Generalization

TL;DR

This dissertation investigates core aspects of GNNs through developing new representation learning techniques based on Graph Shift Operators (GSOs), and introducing generalization-enhancing methods through graph data augmentation, which provides a more principled understanding of the limitations and potential of GNNs.

Abstract

Graph Neural Networks (GNNs) have emerged as powerful tools for learning representations from structured data. Despite their growing popularity and success across various applications, GNNs encounter several challenges that limit their performance. in their generalization, robustness to adversarial perturbations, and the effectiveness of their representation learning capabilities. In this dissertation, I investigate these core aspects through three main contributions: (1) developing new representation learning techniques based on Graph Shift Operators (GSOs, aiming for enhanced performance across various contexts and applications, (2) introducing generalization-enhancing methods through graph data augmentation, and (3) developing more robust GNNs by leveraging orthonormalization techniques and noise-based defenses against adversarial attacks. By addressing these challenges, my work provides a more principled understanding of the limitations and potential of GNNs.
Paper Structure (214 sections, 20 theorems, 237 equations, 21 figures, 63 tables, 6 algorithms)

This paper contains 214 sections, 20 theorems, 237 equations, 21 figures, 63 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Introduction
  3. The Challenges in GNNs
  4. Overview of Contributions
  5. Graph Representation Learning.
  6. GNN's Robustness.
  7. GNN's Generalization.
  8. Preliminaries
  9. Notation
  10. Graph Properties
  11. Heterogeneity
  12. Homophily
  13. Sparsity
  14. Connectivity
  15. Graph Exploraton: Walks and Random Walks
  16. ...and 199 more sections

Key Result

Proposition 2.4.3

Let $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ be a connected, non-empty, finite graph without isolated vertices. We have where $\delta (\mathcal{G})$ and $\Delta (\mathcal{G})$ represent the minimum and maximum degree of the graph, $\lambda_1(\mathbf{L}_{\text{rw}})$ is the normalized spectral gap, i. e. the smallest non-zero eigenvalue of $\mathbf{L}_{\text{rw}}$, and $\varrho_r(H)$ is Equidist

Figures (21)

  • Figure 1: Result for the spectral clustering task on the Cora graph dataset_node_classification with core numbers considered as clusters. We report the values of the Adjusted Mutual Information (AMI) in percentage for different combinations of the exponents $(e_2, e_3)$ in $\mathbf{V}^{e_2} \mathbf{A} \mathbf{V}^{e_3}.$
  • Figure 2: Effect of GCN's depth on sparse and dense subgraphs. The figure shows the performance of GCNs when varying layer depths, and comparing its effectiveness on both sparse and dense subgraphs.
  • Figure 3: Illustration of ADMP-GNN, when the maximum GNN depth is $L=3$.
  • Figure 4: (a) and (b) display $Adv^{\alpha, \beta}_{\epsilon}[f]$ for Cora and OGBN-Arxiv. (c) Robustness guarantees on Cora, where $r_a,r_d$ are respectively the maximum number of adversarial additions and deletions.
  • Figure 5: Illustration of our RobustCRF approach. We use the input graph manifold to generate the structure of the CRF, i.e., $V^{\text{CRF}}, E^{\text{CRF}}$. We use the GNN's predictions to generate the observables $\left \{ Y_a:a \in V^{\text{CRF}} \right \}$, we then run the CRF inference to generate the new GNN's predictions $\{ \widetilde{Y}_a: a \in V^{\text{CRF}} \}$.
  • ...and 16 more figures

Theorems & Definitions (42)

  • Definition 2.3.1: Walk on a Finite Graph
  • Definition 2.3.2: Random Walk on a Finite Graph
  • Definition 2.4.1
  • Definition 2.4.2: Cheeger constant
  • Proposition 2.4.3: Discrete Cheeger inequality, cheeger1970lower
  • Proposition 2.4.4: Discrete Buser inequality, buser1982note
  • Proposition 2.4.5: Spectral Properties of the Unnormalised Laplacian Matrix $\mathbf{L}$
  • Proposition 3.3.1
  • Proposition 3.3.2
  • Definition 3.3.3
  • ...and 32 more