Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Normalize Then Propagate: Efficient Homophilous Regularization for Few-shot Semi-Supervised Node Classification

Baoming Zhang, MingCai Chen, Jianqing Song, Shuangjie Li, Jie Zhang, Chongjun Wang

TL;DR

The paper tackles few-shot semi-supervised node classification by exploiting unlabeled nodes through homophily-based supervision. It introduces NormProp, which maps node features to unit hyperspheres via a two-layer MLP and then applies a fixed low-pass propagation, enabling classification by cosine similarity to hyperspherical prototypes. A key contribution is decoupling direction (class information) from Euclidean norm (consistency) and adding homophilous regularization with an upper-bound on the propagated norm, guided by a warm-up training schedule. Empirically, NormProp achieves state-of-the-art performance under low-label regimes with lower computational cost, including strong results on ogbn-arxiv, demonstrating scalability and efficiency for few-shot semi-supervised node classification.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable ability in semi-supervised node classification. However, most existing GNNs rely heavily on a large amount of labeled data for training, which is labor-intensive and requires extensive domain knowledge. In this paper, we first analyze the restrictions of GNNs generalization from the perspective of supervision signals in the context of few-shot semi-supervised node classification. To address these challenges, we propose a novel algorithm named NormProp, which utilizes the homophily assumption of unlabeled nodes to generate additional supervision signals, thereby enhancing the generalization against label scarcity. The key idea is to efficiently capture both the class information and the consistency of aggregation during message passing, via decoupling the direction and Euclidean norm of node representations. Moreover, we conduct a theoretical analysis to determine the upper bound of Euclidean norm, and then propose homophilous regularization to constraint the consistency of unlabeled nodes. Extensive experiments demonstrate that NormProp achieve state-of-the-art performance under low-label rate scenarios with low computational complexity.

Normalize Then Propagate: Efficient Homophilous Regularization for Few-shot Semi-Supervised Node Classification

TL;DR

The paper tackles few-shot semi-supervised node classification by exploiting unlabeled nodes through homophily-based supervision. It introduces NormProp, which maps node features to unit hyperspheres via a two-layer MLP and then applies a fixed low-pass propagation, enabling classification by cosine similarity to hyperspherical prototypes. A key contribution is decoupling direction (class information) from Euclidean norm (consistency) and adding homophilous regularization with an upper-bound on the propagated norm, guided by a warm-up training schedule. Empirically, NormProp achieves state-of-the-art performance under low-label regimes with lower computational cost, including strong results on ogbn-arxiv, demonstrating scalability and efficiency for few-shot semi-supervised node classification.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable ability in semi-supervised node classification. However, most existing GNNs rely heavily on a large amount of labeled data for training, which is labor-intensive and requires extensive domain knowledge. In this paper, we first analyze the restrictions of GNNs generalization from the perspective of supervision signals in the context of few-shot semi-supervised node classification. To address these challenges, we propose a novel algorithm named NormProp, which utilizes the homophily assumption of unlabeled nodes to generate additional supervision signals, thereby enhancing the generalization against label scarcity. The key idea is to efficiently capture both the class information and the consistency of aggregation during message passing, via decoupling the direction and Euclidean norm of node representations. Moreover, we conduct a theoretical analysis to determine the upper bound of Euclidean norm, and then propose homophilous regularization to constraint the consistency of unlabeled nodes. Extensive experiments demonstrate that NormProp achieve state-of-the-art performance under low-label rate scenarios with low computational complexity.
Paper Structure (19 sections, 14 equations, 6 figures, 3 tables)

This paper contains 19 sections, 14 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Works
  3. Notations
  4. Decoupled GCN
  5. Related Works
  6. NormProp: Normalize then Propagate
  7. Homophilous Regularization: Consistent Loss based on Euclidean Norm
  8. Upper Bound of Euclidean Norm
  9. Homophilous Regularization
  10. Reinterpretation of Homophilous Regularization
  11. Model Implementation and Training
  12. Implementation of NormProp
  13. Model Training
  14. Experiments
  15. Experiment Setup
  16. ...and 4 more sections

Figures (6)

  • Figure 1: The source of the loss signal for a two-layer GNN. Suppose that there is only one labeled node in the graph, and that the graph consists of two connected components.
  • Figure 2: An illustration of the proposed NormProp (Normalize then Propagate).
  • Figure 3: Impact of hyper-parameter $\lambda$.
  • Figure 4: Impact of hyper-parameter threshold $\tau$.
  • Figure 5: Global bias of NormProp on Cora.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Definition 1: Normed Vector Space
  • Definition 2: Consistent Metric