Residual Gated Graph ConvNets

TL;DR

The paper systematically compares graph RNNs and graph ConvNets for variable-length graphs and introduces Residual Gated Graph ConvNets (with edge gates and skip connections). Through analytically controlled experiments on subgraph matching and semi-supervised clustering, the authors show ConvNets outperform RNNs, especially as depth increases, and demonstrate substantial speed advantages over recurrent approaches. The residual gating mechanism is shown to provide a notable performance boost when stacking multiple layers, making deep graph networks practical. Overall, the work highlights the importance of gating and residual design in enabling effective deep learning on graphs and points toward applications in chemistry, physics, and neuroscience.

Abstract

Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks. Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced. In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks. We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size. Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures. Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs. Graph ConvNets are also 36% more accurate than variational (non-learning) techniques. Finally, the most effective graph ConvNet architecture uses gated edges and residuality. Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.

Figures (6)

• Figure 1: Generic feature representation $h_i$ of vertex $i$ on a graph RNN (a) and a graph convNet (b).
• Figure 2: Graph learning tasks.
• Figure 3: Subgraph matching: First row studies shallow networks w.r.t. noise. Second row investigates multilayer graph networks. Third row reports graph architectures w.r.t. budget.
• Figure 4: Influence of hyper-parameter $T$ on RNN architectures. Left figure is for graph matching, middle figure for semi-supervised clustering, and right figure are the batch time for the clustering task (same trend for matching).
• Figure 5: Semi-supervised clustering: First row reports shallow networks w.r.t. noise $q$. Second row shows multilayer graph networks w.r.t. $L$. Third row is about graph architectures w.r.t. budget $B$.