Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual Mamba for Node-Specific Representation Learning: Tackling Over-Smoothing with Selective State Space Modeling

Xin He, Yili Wang, Yiwei Dai, Xin Wang

TL;DR

This work tackles over-smoothing in deep Graph Neural Networks by introducing DMbaGCN, a dual Mamba-based framework that models node-specific evolution with Local State-Evolution Mamba (LSEMba) and global context with Global Context-Aware Mamba (GCAMba). LSEMba captures progressive, node-specific dynamics across layers through selective state-space modeling of multi-hop neighborhood information, while GCAMba injects global information using bidirectional Mamba with linear-time complexity. The combination yields node representations that remain discriminative in deep networks and scales efficiently, outperforming both deep GNNs and graph Transformer baselines on multiple benchmarks. The work demonstrates that integrating Mamba into GNNs offers a practical, efficient route to mitigating over-smoothing while maintaining scalability.

Abstract

Over-smoothing remains a fundamental challenge in deep Graph Neural Networks (GNNs), where repeated message passing causes node representations to become indistinguishable. While existing solutions, such as residual connections and skip layers, alleviate this issue to some extent, they fail to explicitly model how node representations evolve in a node-specific and progressive manner across layers. Moreover, these methods do not take global information into account, which is also crucial for mitigating the over-smoothing problem. To address the aforementioned issues, in this work, we propose a Dual Mamba-enhanced Graph Convolutional Network (DMbaGCN), which is a novel framework that integrates Mamba into GNNs to address over-smoothing from both local and global perspectives. DMbaGCN consists of two modules: the Local State-Evolution Mamba (LSEMba) for local neighborhood aggregation and utilizing Mamba's selective state space modeling to capture node-specific representation dynamics across layers, and the Global Context-Aware Mamba (GCAMba) that leverages Mamba's global attention capabilities to incorporate global context for each node. By combining these components, DMbaGCN enhances node discriminability in deep GNNs, thereby mitigating over-smoothing. Extensive experiments on multiple benchmarks demonstrate the effectiveness and efficiency of our method.

Dual Mamba for Node-Specific Representation Learning: Tackling Over-Smoothing with Selective State Space Modeling

TL;DR

This work tackles over-smoothing in deep Graph Neural Networks by introducing DMbaGCN, a dual Mamba-based framework that models node-specific evolution with Local State-Evolution Mamba (LSEMba) and global context with Global Context-Aware Mamba (GCAMba). LSEMba captures progressive, node-specific dynamics across layers through selective state-space modeling of multi-hop neighborhood information, while GCAMba injects global information using bidirectional Mamba with linear-time complexity. The combination yields node representations that remain discriminative in deep networks and scales efficiently, outperforming both deep GNNs and graph Transformer baselines on multiple benchmarks. The work demonstrates that integrating Mamba into GNNs offers a practical, efficient route to mitigating over-smoothing while maintaining scalability.

Abstract

Over-smoothing remains a fundamental challenge in deep Graph Neural Networks (GNNs), where repeated message passing causes node representations to become indistinguishable. While existing solutions, such as residual connections and skip layers, alleviate this issue to some extent, they fail to explicitly model how node representations evolve in a node-specific and progressive manner across layers. Moreover, these methods do not take global information into account, which is also crucial for mitigating the over-smoothing problem. To address the aforementioned issues, in this work, we propose a Dual Mamba-enhanced Graph Convolutional Network (DMbaGCN), which is a novel framework that integrates Mamba into GNNs to address over-smoothing from both local and global perspectives. DMbaGCN consists of two modules: the Local State-Evolution Mamba (LSEMba) for local neighborhood aggregation and utilizing Mamba's selective state space modeling to capture node-specific representation dynamics across layers, and the Global Context-Aware Mamba (GCAMba) that leverages Mamba's global attention capabilities to incorporate global context for each node. By combining these components, DMbaGCN enhances node discriminability in deep GNNs, thereby mitigating over-smoothing. Extensive experiments on multiple benchmarks demonstrate the effectiveness and efficiency of our method.

Paper Structure

This paper contains 25 sections, 14 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Over-smoothing in GNNs
  4. Graph Transformer
  5. Graph Mamba
  6. Preliminary
  7. Problem Statement
  8. General Graph Mamba
  9. DMbaGCN
  10. Local State-Evolution Mamba (LSEMba)
  11. Global Context-Aware Mamba (GCAMba)
  12. Experiment
  13. Experimental Setting
  14. Performance Evaluation (Q1)
  15. Layer-Wise Performance Trends (Q2)
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Comparison between GCN and Mamba (top), and comparison between Transformer and Mamba (bottom).
  • Figure 2: The Framework of DMbaGCN. LSEMba models the evolution of node representations across GNN layers using Mamba's selective state space modeling. GCAMba aggregates global information for each node through bidirectional Mamba.
  • Figure 3: Comparison of Time and Memory Consumption.
  • Figure 4: Effect of Hyperparameters $\alpha$ and $\beta$ on Model Performance