Hyperbolic Hypergraph Neural Networks for Multi-Relational Knowledge Hypergraph Representation

TL;DR

This work tackles the challenge of learning representations for multi-relational knowledge hypergraphs by treating hyperedges as structured instances with order-sensitive roles. It introduces H$^2$GNN, a fully hyperbolic encoder that employs a two-stage hyper-star message passing to fuse adjacent hyperedges, hyper-relations, and position-aware information, operating in the Lorentz space with centroid-based aggregations and composition. The model demonstrates state-of-the-art performance on node classification and knowledge hypergraph link prediction across multiple datasets, with ablations confirming the importance of hyperbolic operations and position-aware components. By preserving hierarchical structure in hyperbolic space, H$^2$GNN offers a scalable, structure-aware approach for complex hypergraph reasoning with practical impact on tasks requiring high-order relational reasoning.

Abstract

Knowledge hypergraphs generalize knowledge graphs using hyperedges to connect multiple entities and depict complicated relations. Existing methods either transform hyperedges into an easier-to-handle set of binary relations or view hyperedges as isolated and ignore their adjacencies. Both approaches have information loss and may potentially lead to the creation of sub-optimal models. To fix these issues, we propose the Hyperbolic Hypergraph Neural Network (H2GNN), whose essential component is the hyper-star message passing, a novel scheme motivated by a lossless expansion of hyperedges into hierarchies. It implements a direct embedding that consciously incorporates adjacent entities, hyper-relations, and entity position-aware information. As the name suggests, H2GNN operates in the hyperbolic space, which is more adept at capturing the tree-like hierarchy. We compare H2GNN with 15 baselines on knowledge hypergraphs, and it outperforms state-of-the-art approaches in both node classification and link prediction tasks.

Figures (6)

• Figure 1: Considering the tuple (Roster, Bucks, Guard, Jrue Holiday) and (Award, Bucks, champion, Season 20-21). We integrate position-aware information into the knowledge hypergraph by introducing relations like Roster-1, Roster-2, Roster-3, Award-1, Award-2, and Award-3. These relations correspond to their order within the tuple and hyperedges and the expansion results in a hierarchical structure.
• Figure 2: Demonstration of instantiated hierarchies in Euclidean and hyperbolic space. The entity 330m denoted by the green circle represents the height of Eiffel Tower and Gothic architecture, highlighted by the orange circle, signifies the architecture style of Santa Maria delle Grazie. These two entities, despite being the $4$-hop neighbors of Mona Lisa, bear little relevance to each other. Therefore, they should be depicted as somewhat distanced on the graph to reflect their weak romantic correlation.
• Figure 3: Visualization $2$-dimensional representations on the DBLP dataset in both Euclidean and Lorentz spaces. Each point denotes an individual author, with color differentiation illustrating their respective labels.
• Figure 4: Illustration showcasing the H$^2$GNN architecture with operations executed in Lorentz space through the Exponential Map. The two-stage hyper-star message passing, denoted as $\phi_1$ and $\phi_2$, is intricately designed for knowledge hypergraphs. In the function $\phi_1$, the aggregation operation refines hyperedge embeddings, exemplified through the updating process of hyperedges like Roster and Award. Transitioning to $\phi_2$, node embeddings undergo modification by integrating aggregated hyperedge embeddings, hyper-relation embeddings, and position-aware information. Before $\phi_2$, these components individually experience a linear transformation as demonstrated by the matrices $\textbf{W}_h$, $\textbf{W}_r$, and $\textbf{W}_p$. A specific instantiation of the node update process is demonstrated for the node Bucks. The decoder incorporates m-DistMult, mTransH, and HsimplE for link prediction, while Softmax is applied to node classification tasks.
• Figure 5: Comparison experiments: encoding the hypergraph structure information with different methods for the same m-DistMult decoding model.