IGNN-Solver: A Graph Neural Solver for Implicit Graph Neural Networks

TL;DR

This work tackles the slow inference of implicit graph neural networks (IGNNs) by introducing IGNN-Solver, a learnable, graph-aware solver that replaces conventional fixed-point iterations with a tiny graph neural network-based updater. The solver uses a generalized Anderson Acceleration framework parameterized by a small GNN, enhanced with an initialized start point and compressed residual histories, plus graph sparsification and storage compression for large-scale graphs. Training is conducted in an unsupervised, solver-centric manner using initializer, reconstruction, and auxiliary losses, while keeping the IGNN parameters frozen during solver training. Empirically, IGNN-Solver delivers 1.5× to 8× inference speedups across nine real-world datasets (including four Open Graph Benchmark graphs) with minimal training overhead and often improved accuracy, enabling scalable deployment of IGNNs in large-scale applications.

Abstract

Implicit graph neural networks (IGNNs), which exhibit strong expressive power with a single layer, have recently demonstrated remarkable performance in capturing long-range dependencies (LRD) in underlying graphs while effectively mitigating the over-smoothing problem. However, IGNNs rely on computationally expensive fixed-point iterations, which lead to significant speed and scalability limitations, hindering their application to large-scale graphs. To achieve fast fixed-point solving for IGNNs, we propose a novel graph neural solver, IGNN-Solver, which leverages the generalized Anderson Acceleration method, parameterized by a tiny GNN, and learns iterative updates as a graph-dependent temporal process. To improve effectiveness on large-scale graph tasks, we further integrate sparsification and storage compression methods, specifically tailored for the IGNN-Solver, into its design. Extensive experiments demonstrate that the IGNN-Solver significantly accelerates inference on both small- and large-scale tasks, achieving a $1.5\times$ to $8\times$ speedup without sacrificing accuracy. This advantage becomes more pronounced as the graph scale grows, facilitating its large-scale deployment in real-world applications. The code to reproduce our results is available at https://github.com/landrarwolf/IGNN-Solver.

Figures (9)

• Figure 1: Comparison of our IGNN-Solver and the generic Anderson solver at each iteration. (a) The canonical Anderson solver performs local least-squares fitting at each iteration, with a fixed $\beta = \beta^{[k]}$ set to a constant. (b) Our learnable IGNN-Solver improves and incorporates learnable iterative updates for enhanced convergence via tiny GNN.
• Figure 2: Speed-accuracy tradeoff curves comparing the inference time of IGNN with IGNN-Solver and its variants IGNN-Solver (NN) and IGNN-Solver (AA), across four large-scale datasets: Amazon-all (\ref{['t_Amazon-all']}), Reddit (\ref{['t_Reddit']}), ogbn-arxiv (\ref{['t_ogbn-arxiv']}) and ogbn-products (\ref{['t_ogbn-products']}). Each plot shows the average performance over five independent runs under identical experimental conditions.
• Figure 3: Speed-accuracy tradeoff curves comparing the inference time of IGNN with IGNN-Solverand its variants IGNN-Solver (NN) and IGNN-Solver (AA), across five small-scale datasets: Citeseer (\ref{['t_Citeseer']}), ACM (\ref{['t_ACM']}), CoraFull (\ref{['t_CoraFull']}), BlogCatalog (\ref{['t_BlogCatalog']}) and Flickr (\ref{['t_Flickr']}). Each plot represents the average performance over five independent runs under identical experimental conditions.
• Figure 4: Relative parameter size and training time of the IGNN-Solver on the small Citeseer dataset and the large Amazon-All dataset, with consistent patterns observed across other datasets.
• Figure 5: Comparison of convergence curves across eight datasets, highlighting the substantial acceleration in convergence achieved by our IGNN-Solver.

Theorems & Definitions (1)

• Remark 1: Well-posedness of IGNN-Solver