Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mamba-Based Graph Convolutional Networks: Tackling Over-smoothing with Selective State Space

Xin He, Yili Wang, Wenqi Fan, Xu Shen, Xin Juan, Rui Miao, Xin Wang

TL;DR

This work introduces MbaGCN, a Mamba-inspired graph neural network backbone designed to tackle over-smoothing in deep GNNs. By alternating Message Aggregation Layers (MAL) with Selective State Space Transition Layers (S3TL) and adding a Node State Prediction Layer (NSPL), the model adaptively aggregates information across neighborhoods of varying orders and dynamically gates junctions via Gumbel-Softmax based state vectors. Key contributions include the data-driven generation of state matrices, HiPPO-LegS initialization for robust deep propagation, and a principled adjacency modulation mechanism that preserves important local features while compressing redundant higher-order information. Empirical results across eight datasets show strong performance, especially on heterophilic graphs, and ablations highlight the critical roles of HL and IR in maintaining performance as depth grows, establishing MbaGCN as a foundation for deeper, more adaptable GNN architectures.

Abstract

Graph Neural Networks (GNNs) have shown great success in various graph-based learning tasks. However, it often faces the issue of over-smoothing as the model depth increases, which causes all node representations to converge to a single value and become indistinguishable. This issue stems from the inherent limitations of GNNs, which struggle to distinguish the importance of information from different neighborhoods. In this paper, we introduce MbaGCN, a novel graph convolutional architecture that draws inspiration from the Mamba paradigm-originally designed for sequence modeling. MbaGCN presents a new backbone for GNNs, consisting of three key components: the Message Aggregation Layer, the Selective State Space Transition Layer, and the Node State Prediction Layer. These components work in tandem to adaptively aggregate neighborhood information, providing greater flexibility and scalability for deep GNN models. While MbaGCN may not consistently outperform all existing methods on each dataset, it provides a foundational framework that demonstrates the effective integration of the Mamba paradigm into graph representation learning. Through extensive experiments on benchmark datasets, we demonstrate that MbaGCN paves the way for future advancements in graph neural network research.

Mamba-Based Graph Convolutional Networks: Tackling Over-smoothing with Selective State Space

TL;DR

This work introduces MbaGCN, a Mamba-inspired graph neural network backbone designed to tackle over-smoothing in deep GNNs. By alternating Message Aggregation Layers (MAL) with Selective State Space Transition Layers (S3TL) and adding a Node State Prediction Layer (NSPL), the model adaptively aggregates information across neighborhoods of varying orders and dynamically gates junctions via Gumbel-Softmax based state vectors. Key contributions include the data-driven generation of state matrices, HiPPO-LegS initialization for robust deep propagation, and a principled adjacency modulation mechanism that preserves important local features while compressing redundant higher-order information. Empirical results across eight datasets show strong performance, especially on heterophilic graphs, and ablations highlight the critical roles of HL and IR in maintaining performance as depth grows, establishing MbaGCN as a foundation for deeper, more adaptable GNN architectures.

Abstract

Graph Neural Networks (GNNs) have shown great success in various graph-based learning tasks. However, it often faces the issue of over-smoothing as the model depth increases, which causes all node representations to converge to a single value and become indistinguishable. This issue stems from the inherent limitations of GNNs, which struggle to distinguish the importance of information from different neighborhoods. In this paper, we introduce MbaGCN, a novel graph convolutional architecture that draws inspiration from the Mamba paradigm-originally designed for sequence modeling. MbaGCN presents a new backbone for GNNs, consisting of three key components: the Message Aggregation Layer, the Selective State Space Transition Layer, and the Node State Prediction Layer. These components work in tandem to adaptively aggregate neighborhood information, providing greater flexibility and scalability for deep GNN models. While MbaGCN may not consistently outperform all existing methods on each dataset, it provides a foundational framework that demonstrates the effective integration of the Mamba paradigm into graph representation learning. Through extensive experiments on benchmark datasets, we demonstrate that MbaGCN paves the way for future advancements in graph neural network research.
Paper Structure (20 sections, 11 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 20 sections, 11 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Wrok
  3. Over-smoothing on Graph Data
  4. Mamba & Mamba with Graph
  5. Preliminary and Background
  6. Mamba
  7. Problem Statement
  8. MbaGCN
  9. Alternating MAL and S3TL
  10. Message Aggregation Layers (MAL)
  11. Selective State Space Transition Layers (S3TL)
  12. Node State Prediction Layer (NSPL)
  13. Total Complexity of MbaGCN
  14. Experiment
  15. Experimental Settings
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Comparison of GCN, MAMBA, and MbaGCN. (a) Traditional GCN aggregates neighborhood information without differentiation. (b) MAMBA retains important information in sequential data through selective state space. (c) MbaGCN enables adaptive aggregation of neighborhood information.
  • Figure 2: The Framework of MbaGCN.
  • Figure 3: Performance of baselines and the proposed MbaGCN with 2/4/6/8/10 layers.