Sheaf4Rec: Sheaf Neural Networks for Graph-based Recommender Systems

TL;DR

This work addresses the limitation of single-vector representations in graph-based recommender systems by introducing Sheaf4Rec, which leverages cellular sheaves to represent users, items, and ratings as vector spaces and applies neural diffusion on a bipartite graph. By combining two embedding tables, a multi-layer sheaf diffusion process, and Bayesian Personalized Ranking loss, the approach yields state-of-the-art performance (notably up to 11.29% relative NDCG@10 improvements) while achieving practical efficiency gains (2.5%–37% faster). Extensive experiments on Facebook Books, Yahoo! Movies, and MovieLens 1M demonstrate robust gains across NDCG, F1, and MRR metrics, with ablations confirming the impact of K, layer depth, and latent dimension. The results highlight the expressive power of the categorical sheaf formalism and point to broad applicability, including potential extensions to next point-of-interest recommendation and other graph-centric domains.

Abstract

Recent advancements in Graph Neural Networks (GNN) have facilitated their widespread adoption in various applications, including recommendation systems. GNNs have proven to be effective in addressing the challenges posed by recommendation systems by efficiently modeling graphs in which nodes represent users or items and edges denote preference relationships. However, current GNN techniques represent nodes by means of a single static vector, which may inadequately capture the intricate complexities of users and items. To overcome these limitations, we propose a solution integrating a cutting-edge model inspired by category theory: Sheaf4Rec. Unlike single vector representations, Sheaf Neural Networks and their corresponding Laplacians represent each node (and edge) using a vector space. Our approach takes advantage from this theory and results in a more comprehensive representation that can be effectively exploited during inference, providing a versatile method applicable to a wide range of graph-related tasks and demonstrating unparalleled performance. Our proposed model exhibits a noteworthy relative improvement of up to 8.53% on F1-Score@10 and an impressive increase of up to 11.29% on NDCG@10, outperforming existing state-of-the-art models such as Neural Graph Collaborative Filtering (NGCF), KGTORe and other recently developed GNN-based models. In addition to its superior predictive capabilities, Sheaf4Rec shows remarkable improvements in terms of efficiency: we observe substantial runtime improvements ranging from 2.5% up to 37% when compared to other GNN-based competitor models, indicating a more efficient way of handling information while achieving better performance. Code is available at https://github.com/antoniopurificato/Sheaf4Rec.

Figures (8)

• Figure 1: The input graph comprises users and items, forming a bipartite structure, with review values on edges. Using two embedding tables, one for the users and one for the items, the graph is transformed to its embedded version.
• Figure 2: The input to the sheaf layers is the embedded graph obtained from the two embedding tables. In the sheaf layers, $0$-cochains are mapped to user embeddings and $1$-cochains to item embeddings. These embeddings are continuously updated. To enhance the representative power of the proposed formalism, multiple sheaf layers are stacked, resulting in better recommendations. The output of the sheaf layers is still a graph, but containing the final representation.
• Figure 3: The final step of the proposed pipelines consists in providing personalized recommendations to users. This involves computing a score $s_{i,j}$ for each user $i$ and item $j$ by performing matrix multiplication between the user embedding $\mathcal{F}^u$ and the item embedding $\mathcal{F}^v$: $(\mathcal{F}^u)^T\mathcal{F}^v$. Finally, scores are ranked to present each user with the most relevant items.
• Figure 4: During training, Sheaf4Rec takes as input the user-item graph, and generates a score matrix $S$ that encapsulates the computed relevance of each item $j$ for each user $i$. To compute the loss function, it is necessary to extract from this matrix the scores corresponding to both positive items ((those with which the user has already interacted) and negative items (those with which the user has not yet interacted). These scores are then used to compute the two terms of the BPR Loss, which is minimized during each training iteration.
• Figure 5: NDCG@K across different values of K for the six models tested on the ML-1M dataset. Sheaf4Rec consistently outperforms other baselines, maintaining its superiority as the parameter K increases.