Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Networks with a Distribution of Parametrized Graphs

See Hian Lee, Feng Ji, Kelin Xia, Wee Peng Tay

TL;DR

This work iteratively determines the distribution of the graphs using a Markov Chain Monte Carlo method, incorporating the principles of PAC-Bayesian theory, and obtains the maximum likelihood estimate of the network parameters in an Expectation-Maximization (EM) framework based on the multiple graphs.

Abstract

Traditionally, graph neural networks have been trained using a single observed graph. However, the observed graph represents only one possible realization. In many applications, the graph may encounter uncertainties, such as having erroneous or missing edges, as well as edge weights that provide little informative value. To address these challenges and capture additional information previously absent in the observed graph, we introduce latent variables to parameterize and generate multiple graphs. We obtain the maximum likelihood estimate of the network parameters in an Expectation-Maximization (EM) framework based on the multiple graphs. Specifically, we iteratively determine the distribution of the graphs using a Markov Chain Monte Carlo (MCMC) method, incorporating the principles of PAC-Bayesian theory. Numerical experiments demonstrate improvements in performance against baseline models on node classification for heterogeneous graphs and graph regression on chemistry datasets.

Graph Neural Networks with a Distribution of Parametrized Graphs

TL;DR

This work iteratively determines the distribution of the graphs using a Markov Chain Monte Carlo method, incorporating the principles of PAC-Bayesian theory, and obtains the maximum likelihood estimate of the network parameters in an Expectation-Maximization (EM) framework based on the multiple graphs.

Abstract

Traditionally, graph neural networks have been trained using a single observed graph. However, the observed graph represents only one possible realization. In many applications, the graph may encounter uncertainties, such as having erroneous or missing edges, as well as edge weights that provide little informative value. To address these challenges and capture additional information previously absent in the observed graph, we introduce latent variables to parameterize and generate multiple graphs. We obtain the maximum likelihood estimate of the network parameters in an Expectation-Maximization (EM) framework based on the multiple graphs. Specifically, we iteratively determine the distribution of the graphs using a Markov Chain Monte Carlo (MCMC) method, incorporating the principles of PAC-Bayesian theory. Numerical experiments demonstrate improvements in performance against baseline models on node classification for heterogeneous graphs and graph regression on chemistry datasets.
Paper Structure (9 sections, 1 theorem, 11 equations, 1 figure)

This paper contains 9 sections, 1 theorem, 11 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Neural Networks
  4. Signal Processing over a Distribution of Graphs
  5. The Problem Formulation
  6. Distributions for Different Graph Types
  7. Maximum Likelihood Estimation
  8. The Proposed Method and Its Derivation
  9. EM for GNN

Key Result

Theorem 1

(Informal) If the parameterization $\lambda \in \Lambda \mapsto S_{\lambda}$ is sufficiently continuous, then the $\mu$-expected feature representation of a GNN model, whose layers are of the form (eq:vsw), changes continuously as the distribution $\mu$ varies.

Figures (1)

  • Figure 1: For a heterogenous graph, we may use a parameter $\lambda$ to control the information transmission rate for each edge type. For example, choosing $\lambda=1$ or $\lambda=0.5$ for the edge type between "disc" and "square" nodes yields different weighted graphs. Conversely, in a homogeneous example with 5 initial edges, by choosing $\lambda_1 = \lambda_2 = 0.2$, $20\%$ of the initial and missing edges are randomly removed and added, potentially forming a "pentagon".

Theorems & Definitions (3)

  • Theorem 1
  • Example 1
  • Remark 1