Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SIGN: Scalable Inception Graph Neural Networks

Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, Federico Monti

TL;DR

This paper tackles the scalability of graph neural networks by introducing SIGN, a sampling-free architecture that uses an inception-style module of multiple precomputable diffusion operators to capture multi-scale neighborhood information. SIGN eliminates graph sampling, enabling extremely fast training and inference on web-scale graphs while maintaining competitive accuracy, and even achieving state-of-the-art results on ogbn-papers100M among sampling-free methods. The approach demonstrates strong empirical performance across inductive and transductive tasks and shows significant runtime advantages, underscoring the practical utility of wide, operator-rich local aggregations over deeper architectures for large graphs. The authors also discuss extensions to higher-order structures and note limitations related to the need for precomputation, offering directions for integrating attention mechanisms in scalable ways.

Abstract

Graph representation learning has recently been applied to a broad spectrum of problems ranging from computer graphics and chemistry to high energy physics and social media. The popularity of graph neural networks has sparked interest, both in academia and in industry, in developing methods that scale to very large graphs such as Facebook or Twitter social networks. In most of these approaches, the computational cost is alleviated by a sampling strategy retaining a subset of node neighbors or subgraphs at training time. In this paper we propose a new, efficient and scalable graph deep learning architecture which sidesteps the need for graph sampling by using graph convolutional filters of different size that are amenable to efficient precomputation, allowing extremely fast training and inference. Our architecture allows using different local graph operators (e.g. motif-induced adjacency matrices or Personalized Page Rank diffusion matrix) to best suit the task at hand. We conduct extensive experimental evaluation on various open benchmarks and show that our approach is competitive with other state-of-the-art architectures, while requiring a fraction of the training and inference time. Moreover, we obtain state-of-the-art results on ogbn-papers100M, the largest public graph dataset, with over 110 million nodes and 1.5 billion edges.

SIGN: Scalable Inception Graph Neural Networks

TL;DR

This paper tackles the scalability of graph neural networks by introducing SIGN, a sampling-free architecture that uses an inception-style module of multiple precomputable diffusion operators to capture multi-scale neighborhood information. SIGN eliminates graph sampling, enabling extremely fast training and inference on web-scale graphs while maintaining competitive accuracy, and even achieving state-of-the-art results on ogbn-papers100M among sampling-free methods. The approach demonstrates strong empirical performance across inductive and transductive tasks and shows significant runtime advantages, underscoring the practical utility of wide, operator-rich local aggregations over deeper architectures for large graphs. The authors also discuss extensions to higher-order structures and note limitations related to the need for precomputation, offering directions for integrating attention mechanisms in scalable ways.

Abstract

Graph representation learning has recently been applied to a broad spectrum of problems ranging from computer graphics and chemistry to high energy physics and social media. The popularity of graph neural networks has sparked interest, both in academia and in industry, in developing methods that scale to very large graphs such as Facebook or Twitter social networks. In most of these approaches, the computational cost is alleviated by a sampling strategy retaining a subset of node neighbors or subgraphs at training time. In this paper we propose a new, efficient and scalable graph deep learning architecture which sidesteps the need for graph sampling by using graph convolutional filters of different size that are amenable to efficient precomputation, allowing extremely fast training and inference. Our architecture allows using different local graph operators (e.g. motif-induced adjacency matrices or Personalized Page Rank diffusion matrix) to best suit the task at hand. We conduct extensive experimental evaluation on various open benchmarks and show that our approach is competitive with other state-of-the-art architectures, while requiring a fraction of the training and inference time. Moreover, we obtain state-of-the-art results on ogbn-papers100M, the largest public graph dataset, with over 110 million nodes and 1.5 billion edges.

Paper Structure

This paper contains 38 sections, 5 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Deep learning on graphs
  4. Basic notions
  5. Convolution-like operators on graphs
  6. Spectral methods
  7. ChebNet
  8. GCN
  9. S-GCN
  10. MotifNet
  11. Graph sampling
  12. Node-wise sampling
  13. Layer-wise sampling
  14. Graph-wise sampling
  15. Scalable Inception Graph Neural Networks
  16. ...and 23 more sections

Figures (3)

  • Figure 1: The SIGN architecture for $r$ generic graph filtering operators. $\boldsymbol{\Theta}_{k}$ represents the $k$-th dense layer transforming node-wise features downstream the application of operator $k$, $\mid$ is the concatenation operation and $\boldsymbol{\Omega}$ refers to the dense layer used to compute final predictions.
  • Figure 2: Convergence of different methods on ogbn-products.
  • Figure 3: Normalized frequency distributions for row-wise variations on the diffusion weights of triangle operators over inductive datasets. Variations are measured as the standard deviation on the weight value over original neighborhoods from the test graph.