Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Practice-Friendly LLM-Enhanced Paradigm with Preference Parsing for Sequential Recommendation

Dugang Liu, Shenxian Xian, Xiaolin Lin, Xiaolian Zhang, Hong Zhu, Yuan Fang, Zhen Chen, Zhong Ming

TL;DR

A practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) for SRS is proposed and a new user-level SFT task for collaborative information injection with the assistance of a pre-trained SRS model is designed, which is more efficient and compatible with limited text information.

Abstract

The training paradigm integrating large language models (LLM) is gradually reshaping sequential recommender systems (SRS) and has shown promising results. However, most existing LLM-enhanced methods rely on rich textual information on the item side and instance-level supervised fine-tuning (SFT) to inject collaborative information into LLM, which is inefficient and limited in many applications. To alleviate these problems, this paper proposes a practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) for SRS. Specifically, in the information reconstruction stage, we design a new user-level SFT task for collaborative information injection with the assistance of a pre-trained SRS model, which is more efficient and compatible with limited text information. Our goal is to let LLM learn to reconstruct a corresponding prior preference distribution from each user's interaction sequence, where LLM needs to effectively parse the latent category of each item and the relationship between different items to accomplish this task. In the information augmentation stage, we feed each item into LLM to obtain a set of enhanced embeddings that combine collaborative information and LLM inference capabilities. These embeddings can then be used to help train various future SRS models. Finally, we verify the effectiveness and efficiency of our TSLRec on three SRS benchmark datasets.

A Practice-Friendly LLM-Enhanced Paradigm with Preference Parsing for Sequential Recommendation

TL;DR

A practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) for SRS is proposed and a new user-level SFT task for collaborative information injection with the assistance of a pre-trained SRS model is designed, which is more efficient and compatible with limited text information.

Abstract

The training paradigm integrating large language models (LLM) is gradually reshaping sequential recommender systems (SRS) and has shown promising results. However, most existing LLM-enhanced methods rely on rich textual information on the item side and instance-level supervised fine-tuning (SFT) to inject collaborative information into LLM, which is inefficient and limited in many applications. To alleviate these problems, this paper proposes a practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) for SRS. Specifically, in the information reconstruction stage, we design a new user-level SFT task for collaborative information injection with the assistance of a pre-trained SRS model, which is more efficient and compatible with limited text information. Our goal is to let LLM learn to reconstruct a corresponding prior preference distribution from each user's interaction sequence, where LLM needs to effectively parse the latent category of each item and the relationship between different items to accomplish this task. In the information augmentation stage, we feed each item into LLM to obtain a set of enhanced embeddings that combine collaborative information and LLM inference capabilities. These embeddings can then be used to help train various future SRS models. Finally, we verify the effectiveness and efficiency of our TSLRec on three SRS benchmark datasets.
Paper Structure (14 sections, 4 equations, 6 figures, 2 tables)

This paper contains 14 sections, 4 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. The Proposed Framework
  5. Information Reconstruction Stage
  6. Characterize User Preferences
  7. SFT for Reconstructing User Preferences
  8. Information Augmentation Stage
  9. Experiments
  10. Experiment Settings
  11. RQ1: Performance Comparison
  12. RQ2: Efficiency Analysis
  13. RQ3: In-depth Analysis of P2Rec
  14. Conclusions

Figures (6)

  • Figure 1: An illustration of a typical LLM-enhanced SRS model training architecture.
  • Figure 2: The architecture of our practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) framework.
  • Figure 3: Illustration of the process of characterizing user preferences.
  • Figure 4: The time required to execute LLM in one epoch in the training and inference phases of P2Rec and E4SRec, respectively.
  • Figure 5: The ratio of different comparison cases between the latent classes obtained by our P2Rec and the pre-grouped results.
  • ...and 1 more figures