Hard negative sampling in hyperedge prediction

TL;DR

This work tackles the challenge of negative sampling in hyperedge prediction by introducing hard negative sampling (HNS), which synthesizes difficult negatives in the hyperedge embedding space through positive-sample injection with a tunable strength $\alpha$. By leveraging hypergraph embeddings and an attention-based synthesis mechanism, HNS yields harder negatives that improve classifier discrimination and model robustness across multiple datasets and architectures. The approach is plug-and-play, scalable with mini-batch training and GPU acceleration, and extends beyond a single model to diverse HGNN-based pipelines. Limitations include dependence on embedding quality and the need to tune $\alpha$, with future work proposed on adaptive injection strategies and broader applications in graph learning tasks.

Abstract

Hypergraph, which allows each hyperedge to encompass an arbitrary number of nodes, is a powerful tool for modeling multi-entity interactions. Hyperedge prediction is a fundamental task that aims to predict future hyperedges or identify existent but unobserved hyperedges based on those observed. In link prediction for simple graphs, most observed links are treated as positive samples, while all unobserved links are considered as negative samples. However, this full-sampling strategy is impractical for hyperedge prediction, due to the number of unobserved hyperedges in a hypergraph significantly exceeds the number of observed ones. Therefore, one has to utilize some negative sampling methods to generate negative samples, ensuring their quantity is comparable to that of positive samples. In current hyperedge prediction, randomly selecting negative samples is a routine practice. But through experimental analysis, we discover a critical limitation of random selecting that the generated negative samples are too easily distinguishable from positive samples. This leads to premature convergence of the model and reduces the accuracy of prediction. To overcome this issue, we propose a novel method to generate negative samples, named as hard negative sampling (HNS). Unlike traditional methods that construct negative hyperedges by selecting node sets from the original hypergraph, HNS directly synthesizes negative samples in the hyperedge embedding space, thereby generating more challenging and informative negative samples. Our results demonstrate that HNS significantly enhances both accuracy and robustness of the prediction. Moreover, as a plug-and-play technique, HNS can be easily applied in the training of various hyperedge prediction models based on representation learning.

Figures (8)

• Figure 1: Visualization of hyperedge embeddings in reduced dimensionality. This figure displays distributions of four different types of samples for the Email-Enron dataset, projected into $2D$ space using high-dimensional embeddings obtained through self-supervised learning: (1) Possible Negatives: All potential negative samples with orders ranging from $3$ to $5$; (2) Positives: The set of positive samples; (3) Random Negatives: Negative samples randomly selected from all possible neighbors; and (4) Hard Negatives: Negative samples obtained by the HNS method.
• Figure 2: The flowchart of a learning-based hyperedge prediction framework. This framework consists of four sequential stages: (1) Node Embedding: Using hypergraph neural networks to embed nodes; (2) Hyperedge Embedding: Generating hyperedge embeddings through neighborhood aggregation; (3) Prediction: Classifying hyperedge embeddings to predict the existence of hyperedges; and (4) Loss Calculation and Model Update: Computing the loss between predicted values and true labels, and updating model parameters accordingly.
• Figure 3: Illustration of how to generate negative samples by HNS. This framework operates through three sequential stages: (1) Weight Calculation: Compute the normalized weights between positive and negative sample embeddings, which will be further utilized to determine the contribution of each positive sample to the synthesis process. (2) Positive Sample Synthesis: Generating synthetic positive samples by weighted aggregation of multiple positive samples; and (3) Negative Sample Augmentation: Inject the synthetic positive samples into randomly sampled negative samples with a coefficient $\alpha$ to control the proportion of positive sample information.
• Figure 4: Illustration of the three negative sampling methods for comparison.
• Figure 5: Comparison of model performance under different positive sample selection strategies. The X-axis represents different hypergraph neural networks, and the Y-axis shows the values of AUC. Subplots (1) to (7) represent the experimental results for the seven real hypergraphs.