Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Training-free Graph Neural Networks and the Power of Labels as Features

Ryoma Sato

TL;DR

This work tackles transductive node classification under limited computational resources by proposing training-free graph neural networks (TFGNNs) that incorporate Labels as Features (LaF). It proves that LaF increases the expressive power of GNNs by enabling representation of label propagation, and introduces TFGNNs whose initialization embeds label-propagation dynamics, allowing immediate deployment with optional training for refinement. The authors demonstrate through experiments on multiple datasets that TFGNNs outperform traditional GNNs in the training-free setting and converge rapidly with minimal training, also showing robustness to feature noise. Although not designed for inductive or heterophilous graphs, this approach opens a practical avenue for fast, label-informed graph learning with potential for integration into broader GNN pipelines and future enhancements.

Abstract

We propose training-free graph neural networks (TFGNNs), which can be used without training and can also be improved with optional training, for transductive node classification. We first advocate labels as features (LaF), which is an admissible but not explored technique. We show that LaF provably enhances the expressive power of graph neural networks. We design TFGNNs based on this analysis. In the experiments, we confirm that TFGNNs outperform existing GNNs in the training-free setting and converge with much fewer training iterations than traditional GNNs.

Training-free Graph Neural Networks and the Power of Labels as Features

TL;DR

This work tackles transductive node classification under limited computational resources by proposing training-free graph neural networks (TFGNNs) that incorporate Labels as Features (LaF). It proves that LaF increases the expressive power of GNNs by enabling representation of label propagation, and introduces TFGNNs whose initialization embeds label-propagation dynamics, allowing immediate deployment with optional training for refinement. The authors demonstrate through experiments on multiple datasets that TFGNNs outperform traditional GNNs in the training-free setting and converge rapidly with minimal training, also showing robustness to feature noise. Although not designed for inductive or heterophilous graphs, this approach opens a practical avenue for fast, label-informed graph learning with potential for integration into broader GNN pipelines and future enhancements.

Abstract

We propose training-free graph neural networks (TFGNNs), which can be used without training and can also be improved with optional training, for transductive node classification. We first advocate labels as features (LaF), which is an admissible but not explored technique. We show that LaF provably enhances the expressive power of graph neural networks. We design TFGNNs based on this analysis. In the experiments, we confirm that TFGNNs outperform existing GNNs in the training-free setting and converge with much fewer training iterations than traditional GNNs.
Paper Structure (20 sections, 3 theorems, 18 equations, 4 figures, 1 table)

This paper contains 20 sections, 3 theorems, 18 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Notations
  4. Transductive Node Classification
  5. Graph Neural Networks
  6. LaF is Admissible, but Not Explored Well
  7. LaF Strengthens the Expressive Power of GNNs
  8. Training-free Graph Neural Networks
  9. Experiments
  10. Experimental Setup
  11. TFGNNs Outperform Existing GNNs in Training-free Setting
  12. Deep TFGNNs Perform Better in Training-free Setting
  13. TFGNNs Converge Fast
  14. TFGNNs are Robust to Feature Noise
  15. Related Work
  16. ...and 5 more sections

Key Result

Theorem 4.1

GNNs with LaF can approximate label propagation with any precision. Specifically, there exists a series of GNNs $\{f^{(l)}_{\text{agg}}\}_l$ and $f_{\text{pred}}$ such that for any positive $\varepsilon$, for any connected graph $G = (V, E, {\boldsymbol{X}})$, for any labeled nodes $V_{\text{train}} where $\hat{{\boldsymbol{y}}}^{\text{LP}}$ is the output of label propagation for test node $v$.

Figures (4)

  • Figure 1: Initialization of TFGNNs. The parameters of the last $(1 + |\mathcal{Y}|)$ rows or $|\mathcal{Y}|$ rows are initialized by $0$ or $1$ in a special pattern
  • Figure 2: Deep TFGNNs perform better in the training-free setting. The x-axis is the number of layers, and the y-axis is the accuracy of the models for the Cora dataset in the training-free setting. These results show that deeper TFGNNs perform better in the training-free setting.
  • Figure 3: TFGNNs converge fast. The x-axis is the number of training iterations, and the y-axis is the validation accuracy of the models for the Cora dataset. These results show that TFGNNs in the optional training mode converge much faster than GCNs.
  • Figure 4: TFGNNs are robust to feature noise. The x-axis is the standard deviation of the Gaussian noise added to the node features, and the y-axis is the accuracy of the models for the Cora dataset. Both models are trained. These results show that TFGNNs are more robust to feature noise than GCNs.

Theorems & Definitions (7)

  • Theorem 4.1
  • proof
  • Proposition 4.2
  • proof
  • Definition 5.1: Training-free Model
  • Proposition 5.2
  • proof