Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RecGPT: Generative Personalized Prompts for Sequential Recommendation via ChatGPT Training Paradigm

Yabin Zhang, Wenhui Yu, Erhan Zhang, Xu Chen, Lantao Hu, Peng Jiang, Kun Gai

TL;DR

RecGPT introduces a generative, ID-based sequential recommender that transfers the ChatGPT training paradigm to item sequences. It combines a personalized auto-regressive Transformer pre-training, prompt-tuning to craft user-specific prompts, and an autoregressive two-step recall during inference to capture evolving user preferences. Empirical results on four public datasets and an online A/B test on Kuaishou demonstrate consistent gains over strong baselines, with prompt-tuning and autoregressive recall providing the most significant benefits. This framework offers a scalable, online-friendly approach to personalized recommendations, bridging the gap between conversational LLMs and production-grade SR systems. The work highlights the potential of using generated prompts to enrich representations and improve multi-step recall in real-time systems.

Abstract

ChatGPT has achieved remarkable success in natural language understanding. Considering that recommendation is indeed a conversation between users and the system with items as words, which has similar underlying pattern with ChatGPT, we design a new chat framework in item index level for the recommendation task. Our novelty mainly contains three parts: model, training and inference. For the model part, we adopt Generative Pre-training Transformer (GPT) as the sequential recommendation model and design a user modular to capture personalized information. For the training part, we adopt the two-stage paradigm of ChatGPT, including pre-training and fine-tuning. In the pre-training stage, we train GPT model by auto-regression. In the fine-tuning stage, we train the model with prompts, which include both the newly-generated results from the model and the user's feedback. For the inference part, we predict several user interests as user representations in an autoregressive manner. For each interest vector, we recall several items with the highest similarity and merge the items recalled by all interest vectors into the final result. We conduct experiments with both offline public datasets and online A/B test to demonstrate the effectiveness of our proposed method.

RecGPT: Generative Personalized Prompts for Sequential Recommendation via ChatGPT Training Paradigm

TL;DR

RecGPT introduces a generative, ID-based sequential recommender that transfers the ChatGPT training paradigm to item sequences. It combines a personalized auto-regressive Transformer pre-training, prompt-tuning to craft user-specific prompts, and an autoregressive two-step recall during inference to capture evolving user preferences. Empirical results on four public datasets and an online A/B test on Kuaishou demonstrate consistent gains over strong baselines, with prompt-tuning and autoregressive recall providing the most significant benefits. This framework offers a scalable, online-friendly approach to personalized recommendations, bridging the gap between conversational LLMs and production-grade SR systems. The work highlights the potential of using generated prompts to enrich representations and improve multi-step recall in real-time systems.

Abstract

ChatGPT has achieved remarkable success in natural language understanding. Considering that recommendation is indeed a conversation between users and the system with items as words, which has similar underlying pattern with ChatGPT, we design a new chat framework in item index level for the recommendation task. Our novelty mainly contains three parts: model, training and inference. For the model part, we adopt Generative Pre-training Transformer (GPT) as the sequential recommendation model and design a user modular to capture personalized information. For the training part, we adopt the two-stage paradigm of ChatGPT, including pre-training and fine-tuning. In the pre-training stage, we train GPT model by auto-regression. In the fine-tuning stage, we train the model with prompts, which include both the newly-generated results from the model and the user's feedback. For the inference part, we predict several user interests as user representations in an autoregressive manner. For each interest vector, we recall several items with the highest similarity and merge the items recalled by all interest vectors into the final result. We conduct experiments with both offline public datasets and online A/B test to demonstrate the effectiveness of our proposed method.
Paper Structure (17 sections, 7 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 17 sections, 7 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sequential Recommendation
  4. ChatGPT for Recommendation
  5. Preliminary
  6. Methodology
  7. Overall Framework
  8. Pre-training auto-regressive generative model
  9. Personalized Prompt-tuning
  10. Inference-validating auto-regressive recall
  11. Experiments
  12. Experimental Setup
  13. Performance Comparison (RQ1)
  14. Parameter Analysis (RQ2)
  15. Ablation Study (RQ3)
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Illustration of training mechanisms of different methods for sequential recommendation. (a) Traditional sequential recommendation methods. (b) Generative personalized prompts for sequential recommendation.
  • Figure 2: Overview of our proposed RecGPT framework. (a) Pre-training stage. (b) Conversations between users and GPT by personalized prompts. (c) To validate auto-regressive generative recommendation ability. The path 1 is traditional inference method and A&B is our proposed auto-regressive two-step inference.
  • Figure 3: (a) Model: Generative Pre-training Transformer (GPT). (b) Pre-training framework based on GPT. (c) Fine-tuning framework with personalized prompts.
  • Figure 4: Sensitive of the size of window $K$ in term of HR@10 and NDCG@10.
  • Figure 5: Sensitive of parameter $m\_n$ in term of HR@10 and NDCG@10.