Multi-Granular Attention based Heterogeneous Hypergraph Neural Network

TL;DR

The paper addresses learning node representations in heterogeneous graphs by overcoming the limitations of meta-path–based methods and the over-squashing problem. It introduces MGA-HHN, which constructs a multi-view heterogeneous hypergraph from symmetric meta-paths and applies a dual-level attention mechanism: node-level transformer-based attention within each view and hyperedge-level attention to fuse across views. Through extensive experiments on three real-world datasets, MGA-HHN consistently outperforms state-of-the-art baselines in node classification, clustering, and visualization, demonstrating superior capture of high-order semantics and long-range dependencies. The approach offers a scalable and expressive framework for heterogeneous graph representation learning with potential impact across domains requiring rich semantic modeling.

Abstract

Heterogeneous graph neural networks (HeteGNNs) have demonstrated strong abilities to learn node representations by effectively extracting complex structural and semantic information in heterogeneous graphs. Most of the prevailing HeteGNNs follow the neighborhood aggregation paradigm, leveraging meta-path based message passing to learn latent node representations. However, due to the pairwise nature of meta-paths, these models fail to capture high-order relations among nodes, resulting in suboptimal performance. Additionally, the challenge of ``over-squashing'', where long-range message passing in HeteGNNs leads to severe information distortion, further limits the efficacy of these models. To address these limitations, this paper proposes MGA-HHN, a Multi-Granular Attention based Heterogeneous Hypergraph Neural Network for heterogeneous graph representation learning. MGA-HHN introduces two key innovations: (1) a novel approach for constructing meta-path based heterogeneous hypergraphs that explicitly models higher-order semantic information in heterogeneous graphs through multiple views, and (2) a multi-granular attention mechanism that operates at both the node and hyperedge levels. This mechanism enables the model to capture fine-grained interactions among nodes sharing the same semantic context within a hyperedge type, while preserving the diversity of semantics across different hyperedge types. As such, MGA-HHN effectively mitigates long-range message distortion and generates more expressive node representations. Extensive experiments on real-world benchmark datasets demonstrate that MGA-HHN outperforms state-of-the-art models, showcasing its effectiveness in node classification, node clustering and visualization tasks.

Figures (11)

• Figure 1: An illustration of a heterogeneous graph, meta-paths and hypergraphs. (a) A heterogeneous graph with three types of nodes: Authors ($A$), Papers ($P$), and Venues ($V$). (b) The Author-Paper-Author ($APA$) and Paper-Author-Paper ($PAP$) meta-path schemas. (c) Instances of the Author-Paper-Author ($APA$) and Paper-Author-Paper ($PAP$) meta-paths. (d) Conventional graphs based on the meta-paths $APA$ and $PAP$. (e) Hypergraph based on the meta-paths $APA$ and $PAP$, where each hyperedge (marked by a dashed circle) captures a multivariate relationship among nodes (e.g., authors a1, a2 and a3 coauthor the same paper p1). In contrast, a meta-path based conventional graph decomposes these multivariate relationships into binary and pairwise relations (e.g., Author a1 coauthors with a2, a1 with a3, and a2 with a3).
• Figure 2: Customized heterogeneous hypergraph for specific downstream tasks. (a) An example heterogeneous graph with three types of nodes: Authors ($A$), Papers ($P$) and Venues ($V$). (b) Customized heterogeneous hypergraphs based on task-specific symmetric meta-paths. (c) Multi-view representation of the corresponding task-related heterogeneous hypergraph.
• Figure 3: The overall framework of MGA-HHN. First, a heterogeneous hypergraph is constructed based on the task-related meta-paths with certain target node type determined by a downstream task. Next, for each hyperedge type, node-level attention is employed to learn node representations within a single view. Finally, hyperedge-level attention is used to fuse the representations of the target nodes from multiple views.
• Figure 4: Node-level attention. First, we use $\mathbf{W}^{Q}, \mathbf{W}^{K}, \mathbf{W}^{V}$ to project node features into query $\left ( \mathbf{Q} \right )$, key $\left ( \mathbf{K}\right )$ and value $\left ( \mathbf{V}\right )$ matrices. Then we use $\frac{\mathbf{{K}^{T}} \cdot \mathbf{Q}}{\sqrt{{d}^{\prime}}}$ to obtain the mutual attention between nodes and update it by combining the normalized adjacency matrix $\mathbf{\bar{A}^r}$. Finally, we perform a weighted sum of all the value matrices to aggregate the target node representations $\mathbf{Z}$.
• Figure 5: Hyperedge-level attention. The weights of each hyperedge type are first learned through hyperedge-level attention, which are further normalized by the softmax function. The weighted sum of the semantic-specific node representations are obtained to generate the final node representations.

Theorems & Definitions (5)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5