Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Classifying Nodes in Graphs without GNNs

Daniel Winter, Niv Cohen, Yedid Hoshen

TL;DR

This work tackles node classification without using Graph Neural Networks by analyzing the success of GNN distillation and proposing CoHOp, a fully GNN-free method. CoHOp combines a simple predictor with a consistency loss that leverages neighbor similarities, an iterative pseudo-labeling scheme to exploit unlabeled data, and neighborhood-label histograms to inject local context. It achieves competitive accuracy on seven standard benchmarks, including inductive settings, without training any GNN, highlighting that GNNs’ advantage may stem from sample efficiency and inductive bias that can be emulated with regularization and local information. The results suggest a practical path to fast, scalable node classification that reduces reliance on heavy graph-based models while maintaining strong performance.

Abstract

Graph neural networks (GNNs) are the dominant paradigm for classifying nodes in a graph, but they have several undesirable attributes stemming from their message passing architecture. Recently, distillation methods succeeded in eliminating the use of GNNs at test time but they still require them during training. We perform a careful analysis of the role that GNNs play in distillation methods. This analysis leads us to propose a fully GNN-free approach for node classification, not requiring them at train or test time. Our method consists of three key components: smoothness constraints, pseudo-labeling iterations and neighborhood-label histograms. Our final approach can match the state-of-the-art accuracy on standard popular benchmarks such as citation and co-purchase networks, without training a GNN.

Classifying Nodes in Graphs without GNNs

TL;DR

This work tackles node classification without using Graph Neural Networks by analyzing the success of GNN distillation and proposing CoHOp, a fully GNN-free method. CoHOp combines a simple predictor with a consistency loss that leverages neighbor similarities, an iterative pseudo-labeling scheme to exploit unlabeled data, and neighborhood-label histograms to inject local context. It achieves competitive accuracy on seven standard benchmarks, including inductive settings, without training any GNN, highlighting that GNNs’ advantage may stem from sample efficiency and inductive bias that can be emulated with regularization and local information. The results suggest a practical path to fast, scalable node classification that reduces reliance on heavy graph-based models while maintaining strong performance.

Abstract

Graph neural networks (GNNs) are the dominant paradigm for classifying nodes in a graph, but they have several undesirable attributes stemming from their message passing architecture. Recently, distillation methods succeeded in eliminating the use of GNNs at test time but they still require them during training. We perform a careful analysis of the role that GNNs play in distillation methods. This analysis leads us to propose a fully GNN-free approach for node classification, not requiring them at train or test time. Our method consists of three key components: smoothness constraints, pseudo-labeling iterations and neighborhood-label histograms. Our final approach can match the state-of-the-art accuracy on standard popular benchmarks such as citation and co-purchase networks, without training a GNN.
Paper Structure (21 sections, 10 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 21 sections, 10 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Motivation
  4. CoHOp - Consistency and Histogram Optimization
  5. Prediction Network
  6. Consistency Loss
  7. Pseudo-Labelling Iterations
  8. Histograms of Neighboring Labels
  9. Experiments
  10. Datasets
  11. Results
  12. Inductive Setting
  13. Ablation study
  14. Homophily Prior
  15. Histogram Approximation
  16. ...and 6 more sections

Figures (4)

  • Figure 1: A linear model using node-only features achieves comparable performance to a full GNN on the PubMed dataset. However, its performance degrades quickly as the training set size decreases.
  • Figure 2: Overview of CoHOp . Our method consists of three elements: (1) augmenting the node features by concatenating them with the histogram of nearby node labels (Sec. \ref{['sec:histograms']}). (2) training with consistency loss in addition to the standard cross-entropy classification loss (Sec \ref{['sec:consistency']}). (3) iterative training with smoothed pseudo-labels (Sec. \ref{['sec:PseudoLabelling']}).
  • Figure 3: The model's accuracy is 1.66% higher on average when using the exact formula (Eq. \ref{['eq:exact_histogram']}) for histogram calculation compared to its approximation (Eq. \ref{['eq:approx_histograms']}).
  • Figure 4: Running times of the pre-processing procedure of calculating the histograms using the exact calculation (Eq. \ref{['eq:exact_histogram']}) and using its approximation (Eq. \ref{['eq:approx_histograms']}) as function as the context size.