Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation

Xuqi Mao, Zhenying He, X. Sean Wang

TL;DR

MAPN tackles deep representation learning in large sparse heterogeneous graphs by integrating a Mamba-based asynchronous propagation framework with a state-space grounded semantic aggregation. It splits the approach into node sequence generation via meta-path guided random walks and asynchronous intra- and inter-meta-path aggregation, using a selective state-space model to retain distant, still-relevant information and mitigate over-smoothing. Empirical results across node and graph classification and long-range benchmarks demonstrate MAPN's robust performance gains over strong baselines, with Laplacian positional encoding further boosting long-range tasks. The work advances scalable, high-fidelity heterogeneous graph embeddings and offers practical guidance for building deep HetG models on large datasets.

Abstract

Graph neural networks (GNNs) have become the state of the art for various graph-related tasks and are particularly prominent in heterogeneous graphs (HetGs). However, several issues plague this paradigm: first, the difficulty in fully utilizing long-range information, known as over-squashing; second, the tendency for excessive message-passing layers to produce indistinguishable representations, referred to as over-smoothing; and finally, the inadequacy of conventional MPNNs to train effectively on large sparse graphs. To address these challenges in deep neural networks for large-scale heterogeneous graphs, this paper introduces the Mamba-based Asynchronous Propagation Network (MAPN), which enhances the representation of heterogeneous sparse graphs. MAPN consists of two primary components: node sequence generation and semantic information aggregation. Node sequences are initially generated based on meta-paths through random walks, which serve as the foundation for a spatial state model that extracts essential information from nodes at various distances. It then asynchronously aggregates semantic information across multiple hops and layers, effectively preserving unique node characteristics and mitigating issues related to deep network degradation. Extensive experiments across diverse datasets demonstrate the effectiveness of MAPN in graph embeddings for various downstream tasks underscoring its substantial benefits for graph representation in large sparse heterogeneous graphs.

MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation

TL;DR

MAPN tackles deep representation learning in large sparse heterogeneous graphs by integrating a Mamba-based asynchronous propagation framework with a state-space grounded semantic aggregation. It splits the approach into node sequence generation via meta-path guided random walks and asynchronous intra- and inter-meta-path aggregation, using a selective state-space model to retain distant, still-relevant information and mitigate over-smoothing. Empirical results across node and graph classification and long-range benchmarks demonstrate MAPN's robust performance gains over strong baselines, with Laplacian positional encoding further boosting long-range tasks. The work advances scalable, high-fidelity heterogeneous graph embeddings and offers practical guidance for building deep HetG models on large datasets.

Abstract

Graph neural networks (GNNs) have become the state of the art for various graph-related tasks and are particularly prominent in heterogeneous graphs (HetGs). However, several issues plague this paradigm: first, the difficulty in fully utilizing long-range information, known as over-squashing; second, the tendency for excessive message-passing layers to produce indistinguishable representations, referred to as over-smoothing; and finally, the inadequacy of conventional MPNNs to train effectively on large sparse graphs. To address these challenges in deep neural networks for large-scale heterogeneous graphs, this paper introduces the Mamba-based Asynchronous Propagation Network (MAPN), which enhances the representation of heterogeneous sparse graphs. MAPN consists of two primary components: node sequence generation and semantic information aggregation. Node sequences are initially generated based on meta-paths through random walks, which serve as the foundation for a spatial state model that extracts essential information from nodes at various distances. It then asynchronously aggregates semantic information across multiple hops and layers, effectively preserving unique node characteristics and mitigating issues related to deep network degradation. Extensive experiments across diverse datasets demonstrate the effectiveness of MAPN in graph embeddings for various downstream tasks underscoring its substantial benefits for graph representation in large sparse heterogeneous graphs.

Paper Structure

This paper contains 22 sections, 2 theorems, 13 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Methodology
  4. Node Sequences Generation Module
  5. Structure Information Extaction
  6. Semantic Aggregation Module
  7. Node Information Transformation
  8. Type-Specific Transformation
  9. Node Information Aggregation
  10. Semantics Information Aggregation
  11. Synchronous Semantic Aggregation
  12. Asychronous Semantic Aggregation
  13. Training
  14. Experiments
  15. Experimental Setup
  16. ...and 7 more sections

Key Result

theorem thmcountertheorem

Consider an $L$-layer Semantic Propagation Network, suppose the activation function $\sigma(\cdot)$ has a Lipschitz constant $\alpha$, and the norm of each element in $W$ is bounded, such that $\|W\|_p \leq c$. Within a $K$-regular graph, for any two nodes $a$ and $b$, and any layer range $0 \leq l'

Figures (3)

  • Figure 1: Degration of deep neural networks.
  • Figure 2: Architecture of MAPN.
  • Figure 3: Asynchronous aggregation.

Theorems & Definitions (2)

  • theorem thmcountertheorem
  • theorem thmcountertheorem