Heterogeneous Sheaf Neural Networks

TL;DR

HetSheaf is introduced, a general framework for heterogeneous sheaf neural networks, and a series of heterogeneous sheaf predictors to better encode the data's heterogeneity into the sheaf structure, achieving competitive results whilst being more parameter-efficient.

Abstract

Heterogeneous graphs, with nodes and edges of different types, are commonly used to model relational structures in many real-world applications. Standard Graph Neural Networks (GNNs) struggle to process heterogeneous data due to oversmoothing. Instead, current approaches have focused on accounting for the heterogeneity in the model architecture, leading to increasingly complex models. Inspired by recent work, we propose using cellular sheaves to model the heterogeneity in the graph's underlying topology. Instead of modelling the data as a graph, we represent it as cellular sheaves, which allows us to encode the different data types directly in the data structure, eliminating the need to inject them into the architecture. We introduce HetSheaf, a general framework for heterogeneous sheaf neural networks, and a series of heterogeneous sheaf predictors to better encode the data's heterogeneity into the sheaf structure. Finally, we empirically evaluate HetSheaf on several standard heterogeneous graph benchmarks, achieving competitive results whilst being more parameter-efficient.

Figures (1)

• Figure 1: HetSheaf: a sheaf-based framework for heterogeneous data. (a) In the standard meta-path approach for heterogeneous GNN, a series of domain-specific meta-paths are extracted from the original heterogeneous graph and then fed into an encoder to generate the latent representations. (b) This is in contrast to the HetSheaf pipeline, which uses a sheaf to capture the underlying heterogeneity. First, the node features are preprocessed by projecting them into the same number of input channels using a linear layer. The resulting features are used to infer the sheaf structure: the nodes are projected into their respective node stalks, and the restriction maps are predicted as described in \ref{['sec:hetero-sheaf-predictors']}. Compared to a standard graph, this structure already encompasses all the necessary information about the heterogeneous structure, eliminating the need for handcrafted architectures. The predicted sheaf is further processed using any SheafGNN method. (c) Learning heterogeneous sheaves. In the sheaf predictor, the node features are enriched with information about the nodes and edge type, to infer a sheaf structure conditioned on the heterogeneous information.

Theorems & Definitions (2)

• Definition 1
• Definition 2