Hypergraph Neural Network with State Space Models for Node Classification

TL;DR

This work addresses the limitation of traditional GNNs in capturing role-based node semantics that are not well represented by adjacency alone. It introduces HGMN, a hypergraph neural network that combines two hypergraph constructions with a learnable Mamba-based state-space fusion to integrate role-aware and adjacency-based embeddings, followed by hypergraph convolution and residual propagation. The model demonstrates consistent performance gains on diverse datasets, including OGB benchmarks, and provides interpretability analyses showing the importance of metapath-based role information. Theoretical guarantees on complexity and convergence, plus comprehensive ablations and scalability assessments, establish HGMN as a robust approach for learning richer, higher-order node representations in large-scale graphs.

Abstract

In recent years, graph neural networks (GNNs) have gained significant attention for node classification tasks on graph-structured data. However, traditional GNNs primarily focus on adjacency relationships between nodes, often overlooking the role-based characteristics that can provide complementary insights for learning expressive node representations. Existing frameworks for extracting role-based features are largely unsupervised and often fail to translate effectively into downstream predictive tasks. To address these limitations, we propose a hypergraph neural network with a state space model (HGMN). The model integrates role-aware representations into GNNs by combining hypergraph construction with state-space modeling in a principled manner. HGMN employs hypergraph construction techniques to capture higher-order relationships and leverages a learnable mamba transformer mechanism to fuse role-based and adjacency-based embeddings. By exploring two distinct hypergraph construction strategies, degree-based and neighborhood-based, the framework reinforces connectivity among nodes with structural similarity, thereby enriching the learned representations. Furthermore, the inclusion of hypergraph convolution layers enables the model to account for complex dependencies within hypergraph structures. To alleviate the over-smoothing problem encountered in deeper networks, we incorporate residual connections, which improve stability and promote effective feature propagation across layers. Comprehensive experiments on benchmark datasets including OGB, ACM, DBLP, IIP TerroristRel, Cora, Citeseer, and Pubmed demonstrate that HGMN consistently outperforms strong baselines in node classification tasks. These results support the claim that explicitly incorporating role-based features within a hypergraph framework offers tangible benefits for node classification tasks.

Figures (7)

• Figure 1: Overview of the proposed HGMN. The input graph is first converted into a hypergraph to capture higher-order node relationships. Node features are transformed into adjacency-based ($X_a$) and role-based ($X_r$) embeddings, which are fused via the Mamba module using linear projections, convolution, and a state-space model, producing $X_f$. Fused embeddings pass through hierarchical hypergraph convolution layers with downsampling to generate multi-level representations ($X_1, \dots, X_{last}$), which are aggregated and fed to a classifier for node-level prediction.
• Figure 2: Simple Graph vs. Hypergraph. (a) A traditional graph with edges connecting pairs of nodes and its vertex-edge incidence matrix. (b) A hypergraph where hyperedges can connect multiple nodes simultaneously, with the corresponding incidence matrix. Hypergraphs capture higher-order relationships that simple graphs cannot, motivating the proposed HGMN to model complex node interactions for improved representation learning.
• Figure 3: Illustration of the Mamba transformer mechanism within HGMN. The block takes adjacency-based features ($\hat{X}_a$) and role-based features ($\hat{X}_r$) as inputs, processes them through structured state-space layers with learnable parameters $(A, B, C)$, and adaptively fuses the resulting outputs using attention coefficients $(\hat{y}_a, \hat{y}_r)$. The fused embedding $X_f$ captures both structural and role-aware information, making the mechanism well-suited for integrating higher-order graph dependencies.
• Figure 4: Effect of the learning rate $\lambda$ on the performance of the proposed HGMN model.
• Figure 5: Effect of the embedding dimension on the performance of the proposed HGMN model.

Theorems & Definitions (4)

• Theorem 4.1: Complexity Bound of HGMN
• proof
• Theorem 4.2: Convergence of HGMN
• proof