Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fixed Aggregation Features Can Rival GNNs

Celia Rubio-Madrigal, Rebekka Burkholz

TL;DR

This work questions the necessity of learning neighborhood aggregations in graph neural networks by introducing Fixed Aggregation Features (FAFs), a training-free method that converts local neighborhoods into fixed multi-hop statistics and feeds them to a downstream MLP. By leveraging the Kolmogorov–Arnol'd representation framework, FAFs provide a principled way to encode multivariate neighborhood information, while practical reducers like mean and sum yield strong, interpretable performance. Across 12 of 14 node-classification benchmarks, FAFs rival or surpass classic GNNs and graph transformers, with Minesweeper and Roman Empire as notable exceptions requiring longer-range signals. The results motivate robust tabular baselines, interpretable diagnostics, and benchmarks that genuinely require learning complex aggregations, suggesting a shift toward simpler, efficient, and transparent graph representations for many tasks.

Abstract

Graph neural networks (GNNs) are widely believed to excel at node representation learning through trainable neighborhood aggregations. We challenge this view by introducing Fixed Aggregation Features (FAFs), a training-free approach that transforms graph learning tasks into tabular problems. This simple shift enables the use of well-established tabular methods, offering strong interpretability and the flexibility to deploy diverse classifiers. Across 14 benchmarks, well-tuned multilayer perceptrons trained on FAFs rival or outperform state-of-the-art GNNs and graph transformers on 12 tasks -- often using only mean aggregation. The only exceptions are the Roman Empire and Minesweeper datasets, which typically require unusually deep GNNs. To explain the theoretical possibility of non-trainable aggregations, we connect our findings to Kolmogorov-Arnold representations and discuss when mean aggregation can be sufficient. In conclusion, our results call for (i) richer benchmarks benefiting from learning diverse neighborhood aggregations, (ii) strong tabular baselines as standard, and (iii) employing and advancing tabular models for graph data to gain new insights into related tasks.

Fixed Aggregation Features Can Rival GNNs

TL;DR

This work questions the necessity of learning neighborhood aggregations in graph neural networks by introducing Fixed Aggregation Features (FAFs), a training-free method that converts local neighborhoods into fixed multi-hop statistics and feeds them to a downstream MLP. By leveraging the Kolmogorov–Arnol'd representation framework, FAFs provide a principled way to encode multivariate neighborhood information, while practical reducers like mean and sum yield strong, interpretable performance. Across 12 of 14 node-classification benchmarks, FAFs rival or surpass classic GNNs and graph transformers, with Minesweeper and Roman Empire as notable exceptions requiring longer-range signals. The results motivate robust tabular baselines, interpretable diagnostics, and benchmarks that genuinely require learning complex aggregations, suggesting a shift toward simpler, efficient, and transparent graph representations for many tasks.

Abstract

Graph neural networks (GNNs) are widely believed to excel at node representation learning through trainable neighborhood aggregations. We challenge this view by introducing Fixed Aggregation Features (FAFs), a training-free approach that transforms graph learning tasks into tabular problems. This simple shift enables the use of well-established tabular methods, offering strong interpretability and the flexibility to deploy diverse classifiers. Across 14 benchmarks, well-tuned multilayer perceptrons trained on FAFs rival or outperform state-of-the-art GNNs and graph transformers on 12 tasks -- often using only mean aggregation. The only exceptions are the Roman Empire and Minesweeper datasets, which typically require unusually deep GNNs. To explain the theoretical possibility of non-trainable aggregations, we connect our findings to Kolmogorov-Arnold representations and discuss when mean aggregation can be sufficient. In conclusion, our results call for (i) richer benchmarks benefiting from learning diverse neighborhood aggregations, (ii) strong tabular baselines as standard, and (iii) employing and advancing tabular models for graph data to gain new insights into related tasks.
Paper Structure (22 sections, 3 theorems, 2 equations, 9 figures, 15 tables)

This paper contains 22 sections, 3 theorems, 2 equations, 9 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fixed Aggregation Features
  4. Advantages of Tabular over Graph Representations
  5. Does Aggregation Need Learning?
  6. Experiments
  7. Conclusions
  8. Study of Reducers for Neighborhood Aggregation
  9. Kolmogorov-Arnold Function $\Phi$ and its Continuous Inverse
  10. Proof of Main Theorem
  11. Loss of Information Over Second Hops
  12. Experimental Details
  13. Dataset Details
  14. Hyperparameters
  15. Runtime
  16. ...and 7 more sections

Key Result

Theorem 4.1

Assume the features $\mathcal{X}$ are orthogonal. Then, the function $h(X) = \sum_{x \in X} x$ defined on multisets $X \subseteq \mathcal{X}$ of bounded size is injective. Moreover, any multiset function $f$ can be decomposed as $f(X) = g\!\left(\sum_{x \in X} x\right)$ for some function $g$.

Figures (9)

  • Figure 1: Fixed Aggregation Features (FAFs) are calculated as a pre-processing step, concatenated to the input ($\oplus$), and fed to an MLP. If $\Phi$ is injective, the neighborhood information is preserved. The Kolmogorov-Arnold theorem ensures the existence of such a function.
  • Figure 2: SHAP feature importance for Minesweeper, stacked by hop. Numbers show features' rankings per hop.
  • Figure 3: Functions $\Phi$ (Thm \ref{['thm:ka']})— and its inverse—, mean and std. Circles and square-like panels (a.i, a.ii, b.ii, b.iii, c.i) live in the 2D space, while segments and Cantor sets (a.iii, b.i, c.ii, c.iii) live in 1D. Colors in (a) and (c) are based on angles on 2D, while colors in (b) are based on position.
  • Figure 4: Example of a two-hop neighborhood with one-hot encoded features and sum aggregation.
  • Figure 5: Train, validation, and test accuracy of FAF+MLP versus GCN. (i)
  • ...and 4 more figures

Theorems & Definitions (4)

  • Theorem 4.1: 1-hop aggregation
  • Theorem 4.2: Thm. 2 of SCHMIDTHIEBER2021119
  • Theorem : 1-hop aggregation
  • proof