Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simplifying Graph Convolutional Networks with Redundancy-Free Neighbors

Jielong Lu, Zhihao Wu, Zhiling Cai, Yueyang Pi, Shiping Wang

TL;DR

This work identifies over-aggregation as the core driver of over-smoothing in deep GCNs and proposes RF-GCN, a redundancy-free, hierarchical neighbor aggregation framework. By constructing redundancy-free adjacency matrices and fusing information via a PPR-inspired set of weights, RF-GCN allows direct, non-redundant messaging from higher-order neighbors while preserving self-information. Theoretical analyses define AvgNAT and SAS to quantify neighbor usage and self-attention, showing the limitations of traditional propagation; RF-GCN mitigates these issues and achieves state-of-the-art results across 16 real-world datasets for node and graph classification. The approach balances deep information propagation with discrimination between nodes, offering practical benefits for networks with varying graph structures and heterophily levels.

Abstract

In recent years, Graph Convolutional Networks (GCNs) have gained popularity for their exceptional ability to process graph-structured data. Existing GCN-based approaches typically employ a shallow model architecture due to the over-smoothing phenomenon. Current approaches to mitigating over-smoothing primarily involve adding supplementary components to GCN architectures, such as residual connections and random edge-dropping strategies. However, these improvements toward deep GCNs have achieved only limited success. In this work, we analyze the intrinsic message passing mechanism of GCNs and identify a critical issue: messages originating from high-order neighbors must traverse through low-order neighbors to reach the target node. This repeated reliance on low-order neighbors leads to redundant information aggregation, a phenomenon we term over-aggregation. Our analysis demonstrates that over-aggregation not only introduces significant redundancy but also serves as the fundamental cause of over-smoothing in GCNs.

Simplifying Graph Convolutional Networks with Redundancy-Free Neighbors

TL;DR

This work identifies over-aggregation as the core driver of over-smoothing in deep GCNs and proposes RF-GCN, a redundancy-free, hierarchical neighbor aggregation framework. By constructing redundancy-free adjacency matrices and fusing information via a PPR-inspired set of weights, RF-GCN allows direct, non-redundant messaging from higher-order neighbors while preserving self-information. Theoretical analyses define AvgNAT and SAS to quantify neighbor usage and self-attention, showing the limitations of traditional propagation; RF-GCN mitigates these issues and achieves state-of-the-art results across 16 real-world datasets for node and graph classification. The approach balances deep information propagation with discrimination between nodes, offering practical benefits for networks with varying graph structures and heterophily levels.

Abstract

In recent years, Graph Convolutional Networks (GCNs) have gained popularity for their exceptional ability to process graph-structured data. Existing GCN-based approaches typically employ a shallow model architecture due to the over-smoothing phenomenon. Current approaches to mitigating over-smoothing primarily involve adding supplementary components to GCN architectures, such as residual connections and random edge-dropping strategies. However, these improvements toward deep GCNs have achieved only limited success. In this work, we analyze the intrinsic message passing mechanism of GCNs and identify a critical issue: messages originating from high-order neighbors must traverse through low-order neighbors to reach the target node. This repeated reliance on low-order neighbors leads to redundant information aggregation, a phenomenon we term over-aggregation. Our analysis demonstrates that over-aggregation not only introduces significant redundancy but also serves as the fundamental cause of over-smoothing in GCNs.

Paper Structure

This paper contains 17 sections, 2 theorems, 23 equations, 8 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Convolutional Networks
  4. Over-smoothing Issue
  5. The Proposed Method
  6. Mathematical Notations
  7. Theoretical Analysis
  8. Redundancy-Free Graph Convolutional Network
  9. Model Analysis
  10. Experiments
  11. Datasets
  12. Compared Methods
  13. Parameter Setups
  14. Semi-supervised Classification
  15. Ablation Study
  16. ...and 2 more sections

Key Result

Theorem 1

For a connected graph $G$ with $N$ nodes, $N-1 \leq \mathrm{AvgNAT}(\mathbf{A}, L) < 2^{(N-2)}$.

Figures (8)

  • Figure 1: Comparison of message passing techniques on graphs. Subfigure (a) illustrates the stepwise information transfer from a node to a target node. Subfigure (b) depicts the proposed model that organizes the neighbors of a node into a hierarchical structure, reducing the risk of overutilizing information from intermediate nodes.
  • Figure 2: An overview of the proposed RF-GCN framework includes detailing the assignment of neighborhoods and hierarchical aggregation of target nodes, followed by the PPR fusion of the resulting representations.
  • Figure 3: The visualization of the representations of all compared methods on the BlogCatalog dataset.
  • Figure 4: The convergence curves of training loss values and validate accuracy with RF-GCN on six datasets.
  • Figure 5: Classification accuracy results (%) for different layers on the dataset Cora, Citeseer, and Pubmed.
  • ...and 3 more figures

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Theorem 2
  • proof