Simple GNNs with Low Rank Non-parametric Aggregators

TL;DR

This work revisits recent spectral GNN approaches to semi-supervised node classification and suggests conventional methods such as non-parametric regression are well suited for semi-supervised learning on sparse, directed networks and a variety of other graph types commonly found in SSNC benchmarks.

Abstract

We revisit recent spectral GNN approaches to semi-supervised node classification (SSNC). We posit that state-of-the-art (SOTA) GNN architectures may be over-engineered for common SSNC benchmark datasets (citation networks, page-page networks, etc.). By replacing feature aggregation with a non-parametric learner we are able to streamline the GNN design process and avoid many of the engineering complexities associated with SOTA hyperparameter selection (GNN depth, non-linearity choice, feature dropout probability, etc.). Our empirical experiments suggest conventional methods such as non-parametric regression are well suited for semi-supervised learning on sparse, directed networks and a variety of other graph types commonly found in SSNC benchmarks. Additionally, we bring attention to recent changes in evaluation conventions for SSNC benchmarking and how this may have partially contributed to rising performances over time.

Figures (6)

• Figure 1: Example of learned spectral filter $\hat{f}(\lambda)$ for the CSBM experiment with an unbounded Sobolev kernel ($\gamma = 0.1$).
• Figure 2: Accuracy comparison of the Kernel model for different graph representations $\bm A$ and $\bm D-\bm A$. Shown above is the signed accuracy difference between the adjacency and Laplacian representations. Best performing kernel was selected per dataset.
• Figure 3: LR Kernel performance relative to the full-rank Kernel for different truncation factors $r$. Performance is seen to gradually decline on most datasets as the truncation factor $r$ decreases (that is truncation percentage increases). LR Kernel performance can also be seen to periodically increase above full-rank Kernel performance for the datasets Chameleon (red) and Squirrel (purple).
• Figure 4: Performance homogenization achieved by LR Kernel model on directed networks.
• Figure 5: Accuracy results and uncertainties on the citation datasets using different splits with linear models $\bm{XW}$ and $\bm{AXW}$. "Public" refers to the split introduced by Kipf17. Both "Sparse" and "Public" are single splits, so one cannot associate uncertainty to them.