Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linear-Time Graph Neural Networks for Scalable Recommendations

Jiahao Zhang, Rui Xue, Wenqi Fan, Xin Xu, Qing Li, Jian Pei, Xiaorui Liu

TL;DR

A Linear-Time Graph Neural Network (LTGNN) is proposed to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy.

Abstract

In an era of information explosion, recommender systems are vital tools to deliver personalized recommendations for users. The key of recommender systems is to forecast users' future behaviors based on previous user-item interactions. Due to their strong expressive power of capturing high-order connectivities in user-item interaction data, recent years have witnessed a rising interest in leveraging Graph Neural Networks (GNNs) to boost the prediction performance of recommender systems. Nonetheless, classic Matrix Factorization (MF) and Deep Neural Network (DNN) approaches still play an important role in real-world large-scale recommender systems due to their scalability advantages. Despite the existence of GNN-acceleration solutions, it remains an open question whether GNN-based recommender systems can scale as efficiently as classic MF and DNN methods. In this paper, we propose a Linear-Time Graph Neural Network (LTGNN) to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy. Extensive experiments and ablation studies are presented to validate the effectiveness and scalability of the proposed algorithm. Our implementation based on PyTorch is available.

Linear-Time Graph Neural Networks for Scalable Recommendations

TL;DR

A Linear-Time Graph Neural Network (LTGNN) is proposed to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy.

Abstract

In an era of information explosion, recommender systems are vital tools to deliver personalized recommendations for users. The key of recommender systems is to forecast users' future behaviors based on previous user-item interactions. Due to their strong expressive power of capturing high-order connectivities in user-item interaction data, recent years have witnessed a rising interest in leveraging Graph Neural Networks (GNNs) to boost the prediction performance of recommender systems. Nonetheless, classic Matrix Factorization (MF) and Deep Neural Network (DNN) approaches still play an important role in real-world large-scale recommender systems due to their scalability advantages. Despite the existence of GNN-acceleration solutions, it remains an open question whether GNN-based recommender systems can scale as efficiently as classic MF and DNN methods. In this paper, we propose a Linear-Time Graph Neural Network (LTGNN) to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy. Extensive experiments and ablation studies are presented to validate the effectiveness and scalability of the proposed algorithm. Our implementation based on PyTorch is available.
Paper Structure (21 sections, 19 equations, 5 figures, 8 tables, 1 algorithm)

This paper contains 21 sections, 19 equations, 5 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations and Definitions
  4. Mini-batch Training
  5. GNNs and MF for Recommendations
  6. The Proposed Method
  7. Implicit Modeling for Recommendations
  8. Efficient Variance-Reduced Neighbor Sampling
  9. Experiments
  10. Experimental Settings
  11. Recommendation Performance
  12. Efficiency Analysis
  13. Ablation Study
  14. Related Work
  15. Graph Collaborative Filtering Models for Recommendations
  16. ...and 6 more sections

Figures (5)

  • Figure 1: An illustration of our model architecture. (a) The forward process of our model aims to solve the PPNP fixed-point equation, which expresses an equilibrium state of the embedding propagations, and can be used to capture long-range relations between any pair of nodes regardless of their distance. (b) The PPNP fixed-point equation is solved with a single forward propagation layer, which leverages the historical output embeddings in previous training iterations. (c) The process of efficient variance-reduced neighbor sampling in LTGNN.
  • Figure 2: Performance comparison between LTGNN and LightGCN using different layers on Yelp2018 and Alibaba-iFashion.
  • Figure 3: Performance of a 1-layer LTGNN w.r.t different numbers of sampled neighbors on Yelp2018 and Alibaba-iFashion.
  • Figure 4: The relative error between the model output $\boldsymbol{E}^k_{out}$ and the exact PPNP propagation result $\boldsymbol{E}^k_{PPNP}$ of the embeddings (i.e., $||\boldsymbol{E}^k_{out} - \boldsymbol{E}^k_{PPNP}||_F / ||\boldsymbol{E}^k_{PPNP}||_F$).
  • Figure 5: The effect of hyper-parameter $\alpha$ under Recall@20 and NDCG@20 metrics.