E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation

TL;DR

The paper tackles the difficulty of applying large language models to sequential recommendation by introducing E4SRec, which injects item ID embeddings into a frozen LLM and uses a light, four-stage pipeline (pretrain, instruction-tune, train E4SRec, deploy) to produce predictions over all candidates in a single forward pass. By leveraging ID embeddings extracted from a pretrained SASRec and a minimal set of trainable components (input projection, adapters, and item output projection), E4SRec achieves controllable, efficient, and extensible recommendations while avoiding out-of-range outputs. Extensive experiments on four real-world datasets show consistent improvements over strong baselines, with substantial gains in nDCG and robust performance in sparse data scenarios. The approach offers industrial practicality through near-backbone inference times, tiny additional parameter counts, and straightforward extensibility for new items without retraining the LLM.

Abstract

The recent advancements in Large Language Models (LLMs) have sparked interest in harnessing their potential within recommender systems. Since LLMs are designed for natural language tasks, existing recommendation approaches have predominantly transformed recommendation tasks into open-domain natural language generation tasks. However, this approach necessitates items to possess rich semantic information, often generates out-of-range results, and suffers from notably low efficiency and limited extensibility. Furthermore, practical ID-based recommendation strategies, reliant on a huge number of unique identities (IDs) to represent users and items, have gained prominence in real-world recommender systems due to their effectiveness and efficiency. Nevertheless, the incapacity of LLMs to model IDs presents a formidable challenge when seeking to leverage LLMs for personalized recommendations. In this paper, we introduce an Elegant Effective Efficient Extensible solution for large language models for Sequential Recommendation (E4SRec), which seamlessly integrates LLMs with traditional recommender systems that exclusively utilize IDs to represent items. Specifically, E4SRec takes ID sequences as inputs, ensuring that the generated outputs fall within the candidate lists. Furthermore, E4SRec possesses the capability to generate the entire ranking list in a single forward process, and demands only a minimal set of pluggable parameters, which are trained for each dataset while keeping the entire LLM frozen. We substantiate the effectiveness, efficiency, and extensibility of our proposed E4SRec through comprehensive experiments conducted on four widely-used real-world datasets. The implementation code is accessible at https://github.com/HestiaSky/E4SRec/.

Figures (4)

• Figure 1: Illustration of LLM-based sequential recommendation. The upper part denotes the existing methods that fail to model the IDs and undesirably generate out-of-range results while the lower part denotes E4SRec that can effectively and efficiently handle the IDs.
• Figure 2: Architecture of E4SRec solution. The left part illustrates the structure of E4SRec, including the input layer, the large language model layer and the prediction layer. The upper right part describes the efficient inference process. The lower right part shows the complete solution of E4SRec.
• Figure 3: Ablation study of ID injection and LLM.
• Figure 4: Performance comparison on different user groups with respect to the length of interaction sequences.