Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Architecture Design

TL;DR

This work reframes hypergraph learning through a message-passing lens to define a dynamic, MP-based notion of homophily that accounts for higher-order structure. It introduces the MultiSet framework and the practical MultiSetMixer architecture, which support hyperedge-dependent node representations and unify prior HNNs as special cases, advancing beyond clique-expansion approaches. A novel mini-batching strategy addresses scalability and reveals a connectivity-based distribution shift, enabling efficient processing of large hyperedges. Extensive experiments across diverse datasets show that hypergraph-native designs excel on certain, connectivity-sensitive tasks while exposing dataset-dependent limitations of lifted approaches, and they establish a new link between dynamic homophily measures and model performance. Overall, the paper provides both theoretical and empirical tools to design and evaluate hypergraph neural networks with explicit higher-order connectivity awareness, guiding future benchmarking and architecture development.

Abstract

Most of the current hypergraph learning methodologies and benchmarking datasets in the hypergraph realm are obtained by lifting procedures from their graph analogs, leading to overshadowing specific characteristics of hypergraphs. This paper attempts to confront some pending questions in that regard: Q1 Can the concept of homophily play a crucial role in Hypergraph Neural Networks (HNNs)? Q2 Is there room for improving current HNN architectures by carefully addressing specific characteristics of higher-order networks? Q3 Do existing datasets provide a meaningful benchmark for HNNs? To address them, we first introduce a novel conceptualization of homophily in higher-order networks based on a Message Passing (MP) scheme, unifying both the analytical examination and the modeling of higher-order networks. Further, we investigate some natural, yet mostly unexplored, strategies for processing higher-order structures within HNNs such as keeping hyperedge-dependent node representations, or performing node/hyperedge stochastic samplings, leading us to the most general MP formulation up to date -MultiSet-, as well as to an original architecture design, MultiSetMixer. Finally, we conduct an extensive set of experiments that contextualize our proposals and successfully provide insights about our inquiries.

Figures (8)

• Figure 1: Node Homophily Distribution for CORA-CA (a) and 20Newsgroups (b). The plots depict node homophily scores computed using Equation \ref{['eq:node_homo_k']} at $t=0$, $t=1$, and $t=10$. For each dataset class, points are sorted in ascending order of homophily and visualized sequentially along the x-axis. Horizontal lines represent the mean homophily score for each class, with the numbers above indicating the total number of points in each class.
• Figure 2: Visual representation of the AllSet and MultiSet frameworks.
• Figure 3: Average rankings at different training percentages, with the overall average performance shown on the right. The error bars indicate the standard deviation of the average rankings across multiple datasets.
• Figure 4: Joint visualization of rank dependencies, showing Norm. Acc. versus $\Delta$ Homophily (Eq. \ref{['eq:delta_homophily']} at step $t=1$ and $\mu=0.1$) and CE Homophily wang2023equivariant. Norm. Acc. (Eq. \ref{['eq:normalized_accuracy']}) is assessed for various instances of model A (specified in column titles), with model B being MLP CB. Both axes represent rank values, with lower values indicating better metrics. Arrows denote the rank shift in homophily between CE homophily and $\Delta$ homophily for each dataset.
• Figure 5: Class distribution shift induced by mini-batching: 'Node' represents the original node class distribution, 'Step 1 and 2' the resulting one after sampling both hyperedges and nodes, and 'Step 1' when only sampling hyperedges.

Theorems & Definitions (5)

• Proposition 4.1
• Proposition 4.2
• Proposition 4.3
• Proposition 4.4
• Proposition 4.5