Expressive Higher-Order Link Prediction through Hypergraph Symmetry Breaking

TL;DR

The paper tackles the expressivity gap in higher-order link prediction on hypergraphs caused by symmetry-induced indistinguishability in GWL-1-based hyperGNNs. It introduces a linear-time preprocessing step that identifies L-GWL-1 symmetric induced subhypergraphs and replaces their edges with covering hyperedges to produce a multi-hypergraph, thereby enabling symmetry breaking during training through random augmentation. The authors provide a formal connection between GWL-1 and universal covers, establish guarantees that the augmentation reduces the symmetry group and increases discriminative power under reasonable conditions, and validate the approach with extensive experiments on graph and hypergraph datasets showing improved PR-AUC with negligible computational overhead. Overall, the method yields a practical, scalable way to enhance hypergraph expressivity and higher-order link prediction performance without sacrificing efficiency.

Abstract

A hypergraph consists of a set of nodes along with a collection of subsets of the nodes called hyperedges. Higher-order link prediction is the task of predicting the existence of a missing hyperedge in a hypergraph. A hyperedge representation learned for higher order link prediction is fully expressive when it does not lose distinguishing power up to an isomorphism. Many existing hypergraph representation learners, are bounded in expressive power by the Generalized Weisfeiler Lehman-1 (GWL-1) algorithm, a generalization of the Weisfeiler Lehman-1 algorithm. However, GWL-1 has limited expressive power. In fact, induced subhypergraphs with identical GWL-1 valued nodes are indistinguishable. Furthermore, message passing on hypergraphs can already be computationally expensive, especially on GPU memory. To address these limitations, we devise a preprocessing algorithm that can identify certain regular subhypergraphs exhibiting symmetry. Our preprocessing algorithm runs once with complexity the size of the input hypergraph. During training, we randomly replace subhypergraphs identified by the algorithm with covering hyperedges to break symmetry. We show that our method improves the expressivity of GWL-1. Our extensive experiments also demonstrate the effectiveness of our approach for higher-order link prediction on both graph and hypergraph datasets with negligible change in computation.

Figures (8)

• Figure 1: An illustration of a hypergraph of recipes. The nodes are the ingredients and the hyperedges are the recipes. The task of higher order link prediction is to predict hyperedges in the hypergraph. A negative hyperedge sample would be the dotted hyperedge. The Asian ingredient nodes ($\alpha$) and the European ingredient nodes ($\beta$) form two separate isomorphism classes. However, GWL-1 cannot distinguish between these classes and will predict a false positive for the negative sample.
• Figure 2: An illustration of hypergraph symmetry breaking. (c,d) $3$-regular hypergraphs $C_4^3$, $C_5^3$ with $4$ and $5$ nodes respectively and their corresponding universal covers centered at any hyperedge $(\tilde{\mathcal{B}}_{C^3_4})_{e_{*,*,*}}, (\tilde{\mathcal{B}}_{C^3_5})_{e_{*,*,*}}$ with universal covering maps $p_{\mathcal{B}_{C_4^3}}, p_{\mathcal{B}_{C_5^3}}$. (b,e) the hypergraphs $\hat{C}_4^3, \hat{C}_5^3$, which are $C_4^3,C_5^3$ with $4,5$-sized hyperedges attached to them and their corresponding universal covers and universal covering maps. (a,f) are the corresponding bipartite graphs of $\hat{C}_4^3, \hat{C}_5^3$. (c,d) are indistinguishable by GWL-1 and thus will give identical node values by Theorem \ref{['thm: gwlunivcover']}. On the other hand, (b,e) gives node values which are now sensitive to the the order of the hypergraphs $4,5$, also by Theorem \ref{['thm: gwlunivcover']}.
• Figure 3: An illustration of Algorithm \ref{['alg: cellattach']} for $1$-GWL-1. In (a) a hypergraph is shown. Each node is labeled with a pair. The left part of the pair in Greek alphabet is its isomorphism class. The right part of the pair in Latin alphabet is its $1$-GWL-1 class, which is determined by its neighborhood of hyperedges. In (b) the multi-hypergraph formed by covering the original hypergraph by hyperedges (light blue boxes) which are determined by the connected components of $1$-GWL-1 indistinguishable node sets. The nodes can now be relabeled by $1$-GWL-1. All hyperedges can be assigned learnable weights. For downstream training, the new hyperedges are randomly added and the existing hyperedges within each new hyperedge are randomly dropped.
• Figure 4: Experiment on the relationship between the sizes of connected components of equal GWL-1 node values and the communication between communities.
• Figure 5: An illustration of hypergraph symmetry breaking. (c,d) $3$-regular hypergraphs $C_4^3$, $C_5^3$ with $4$ and $5$ nodes respectively and their corresponding universal covers centered at any hyperedge $(\tilde{\mathcal{B}}_{C^3_4})_{e_{*,*,*}}, (\tilde{\mathcal{B}}_{C^3_5})_{e_{*,*,*}}$ with universal covering maps $p_{\mathcal{B}_{C_4^3}}, p_{\mathcal{B}_{C_5^3}}$. (b,e) the hypergraphs $\hat{C}_4^3, \hat{C}_5^3$, which are $C_4^3,C_5^3$ with $4,5$-sized hyperedges attached to them and their corresponding universal covers and universal covering maps. (a,f) are the corresponding bipartite graphs of $\hat{C}_4^3, \hat{C}_5^3$. (c,d) are indistinguishable by GWL-1 and thus will give identical node values by Theorem \ref{['thm: appendix-gwlunivcover']}. On the other hand, (b,e) gives node values which are now sensitive to the the order of the hypergraphs $4,5$, also by Theorem \ref{['thm: appendix-gwlunivcover']}.

Theorems & Definitions (96)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Definition 2.5: star expansion bipartite graph
• Definition 2.6
• Proposition 2.1
• Definition 2.7
• Definition 2.8
• Definition 2.9