GraphPro: Graph Pre-training and Prompt Learning for Recommendation

TL;DR

A framework called GraphPro is proposed that combines dynamic graph pre-training with prompt learning in an efficient way that allows GNNs to effectively capture both long-term user preferences and short-term behavior changes, resulting in accurate and up-to-date recommendations.

Abstract

GNN-based recommenders have excelled in modeling intricate user-item interactions through multi-hop message passing. However, existing methods often overlook the dynamic nature of evolving user-item interactions, which impedes the adaption to changing user preferences and distribution shifts in newly arriving data. Thus, their scalability and performances in real-world dynamic environments are limited. In this study, we propose GraphPro, a framework that incorporates parameter-efficient and dynamic graph pre-training with prompt learning. This novel combination empowers GNNs to effectively capture both long-term user preferences and short-term behavior dynamics, enabling the delivery of accurate and timely recommendations. Our GraphPro framework addresses the challenge of evolving user preferences by seamlessly integrating a temporal prompt mechanism and a graph-structural prompt learning mechanism into the pre-trained GNN model. The temporal prompt mechanism encodes time information on user-item interaction, allowing the model to naturally capture temporal context, while the graph-structural prompt learning mechanism enables the transfer of pre-trained knowledge to adapt to behavior dynamics without the need for continuous incremental training. We further bring in a dynamic evaluation setting for recommendation to mimic real-world dynamic scenarios and bridge the offline-online gap to a better level. Our extensive experiments including a large-scale industrial deployment showcases the lightweight plug-in scalability of our GraphPro when integrated with various state-of-the-art recommenders, emphasizing the advantages of GraphPro in terms of effectiveness, robustness and efficiency. The implementation details and source code of our GraphPro are available in the repository at https://github.com/HKUDS/GraphPro

Figures (8)

• Figure 1: Our dynamic recommendation setting compared to the vanilla single-graph training in existing methods.
• Figure 2: Overall framework of GraphPro.
• Figure 3: Key component ablation study for fine-tuning stage. Y-axis denotes performance metrics on the left and epochs (displayed as $\triangledown$) for convergence on the right.
• Figure 4: Ablation study for pretrained models.
• Figure 5: Evaluation performance for tuned and untuned users on Amazon compared with the best baseline, GPF.