Ada-HGNN: Adaptive Sampling for Scalable Hypergraph Neural Networks

TL;DR

Ada-HGNN tackles the scalability challenge of HGNNs by introducing a one-step adaptive sampling strategy tailored to hypergraphs, complemented by Random Hyperedge Augmentation and a pretraining MLP module. The method converts the hypergraph into a joint node-hyperedge expansion $ ilde{\text{G}}$ and employs Generative Flow Networks (GFlowNets) to learn diverse, reward-guided sampling policies, with a trajectory balance objective guiding optimization. Empirical results across seven real-world datasets show substantial memory and computation reductions while maintaining or surpassing baseline accuracy; ablations confirm the benefits of RHA and MLP initialization. This work enables scalable, robust HGNNs for large-scale applications and provides a public code release.

Abstract

Hypergraphs serve as an effective model for depicting complex connections in various real-world scenarios, from social to biological networks. The development of Hypergraph Neural Networks (HGNNs) has emerged as a valuable method to manage the intricate associations in data, though scalability is a notable challenge due to memory limitations. In this study, we introduce a new adaptive sampling strategy specifically designed for hypergraphs, which tackles their unique complexities in an efficient manner. We also present a Random Hyperedge Augmentation (RHA) technique and an additional Multilayer Perceptron (MLP) module to improve the robustness and generalization capabilities of our approach. Thorough experiments with real-world datasets have proven the effectiveness of our method, markedly reducing computational and memory demands while maintaining performance levels akin to conventional HGNNs and other baseline models. This research paves the way for improving both the scalability and efficacy of HGNNs in extensive applications. We will also make our codebase publicly accessible.

Figures (6)

• Figure 1: A visual comparison between the standard 2-hop HGNN computation sampling and our proposed adaptive sampling method. (a) This diagram depicts the conventional HGNN computation, which sequentially processes information from node to hyperedge and back to node, necessitating a two-step sampling strategy that accounts for both nodes and their connecting hyperedges. (b) Illustrates our proposed method, which streamlines this process by integrating nodes with their hyperedges into a singular vertex representation and employing adaptive sampling of neighbor nodes that are relevant to the task at hand. This approach aims to reduce memory overhead while preserving task performance.
• Figure 2: Schematic illustration of the Adaptive Hypergraph Sampling and Learning Process. The workflow begins with a Hyperedge Augmentation and Hyperedge-dependent Expansion, followed by probabilistic node sampling via the GFlowNet policy network. A pre-trained Multi-Layer Perceptron (MLP) can be deployed for initialization of the GNN classifier, which is then processed for gradient update. The GFlowNet is trained using GNN classifier loss as a reward and minimizes log-partition variance, based on trajectory feedback.
• Figure 3: Memory Comparison of our method with 512/1024 batch size, HGNN, and Clique Expension with GCN(CEGNN).
• Figure 4: Mean and standard deviation entropy of nodes including probability output by adaptive sampling network. Small means represents Ada-GT learns strong preference to include/exclude some nodes
• Figure 5: Node (18,18) and its subset of neighbors from the ModelNet40 dataset. The Adaptive-HGNN model identifies which neighbors are informative. The numbers in the nodes represent the hypernode_id and hyperedge_id. Colors indicate the class of each node