Geom-GCN: Geometric Graph Convolutional Networks

TL;DR

Geom-GCN tackles two core weaknesses of traditional MPNNs: loss of discriminative structural information in neighborhoods and inadequate modeling of long-range dependencies in disassortative graphs. It introduces a geometric aggregation scheme with node embedding, dual structural neighborhoods (graph and latent space), and a bi-level permutation-invariant aggregator, instantiated with Isomap, Poincaré, or struc2vec embeddings. The approach achieves state-of-the-art results on nine graph datasets and is supported by ablation and analysis showing the value of latent-space neighborhoods and embedding choices. By mapping graphs to continuous geometric spaces, it preserves topology patterns like hierarchy and long-range similarity, enabling more discriminative representations and broader applicability; future work aims at end-to-end embedding selection and scalability improvements.

Abstract

Message-passing neural networks (MPNNs) have been successfully applied to representation learning on graphs in a variety of real-world applications. However, two fundamental weaknesses of MPNNs' aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs. Few studies have noticed the weaknesses from different perspectives. From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses. The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph. The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation. We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN (Geometric Graph Convolutional Networks), to perform transductive learning on graphs. Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs. Code is available at https://github.com/graphdml-uiuc-jlu/geom-gcn.

Figures (3)

• Figure 1: An illustration of the geometric aggregation scheme. A1-A2 The original graph is mapped to a latent continuous space. B1-B2 The structural neighborhood. All adjacent nodes lie in a small region around a center node in B1 for visualization. In B2, the neighborhood in the graph contains all adjacent nodes in graph; the neighborhood in the latent space contains the nodes within the dashed circle whose radius is $\rho$. The relational operator $\tau$ is illustrated by a colorful $3\times3$ grid where each unit is corresponding to a geometric relationship to the red target node. C Bi-level aggregation on the structural neighborhood. Dashed and solid arrows denote the low-level and high-level aggregation, respectively. Blue and green arrows denote the aggregation on the neighborhood in the graph and the latent space, respectively.
• Figure 2: An illustration to distinguish non-isomorphic graphs by proposed structural neighborhood.
• Figure 3: (a) Running time comparison. GCN, GAT, and Geom-GCN both run 500 epochs, and $y$ axis is the log seconds. GCN is the fastest, and GAT and Geom-GCN are on the same level. (b) A visualization for the feature representations of Cora obtained from Geom-GCN-P in a 2-D space. Node colors denote node labels. There are two obvious patterns, nodes with the same label exhibit a spatial clustering and all nodes distribute radially. The radial pattern indicates graph's hierarchy learned by Poincare embedding.

Theorems & Definitions (2)

• proof
• proof