Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond

TL;DR

A novel taxonomy is introduced that originates from the intrinsic essence of recommendation, delving into the application of large language model-based recommendation systems and their industrial implementation, and proposes a three-tier structure that more accurately reflects the developmental progression of recommendation systems from research to practical implementation.

Abstract

Large language models (LLMs) have not only revolutionized the field of natural language processing (NLP) but also have the potential to bring a paradigm shift in many other fields due to their remarkable abilities of language understanding, as well as impressive generalization capabilities and reasoning skills. As a result, recent studies have actively attempted to harness the power of LLMs to improve recommender systems, and it is imperative to thoroughly review the recent advances and challenges of LLM-based recommender systems. Unlike existing work, this survey does not merely analyze the classifications of LLM-based recommendation systems according to the technical framework of LLMs. Instead, it investigates how LLMs can better serve recommendation tasks from the perspective of the recommender system community, thus enhancing the integration of large language models into the research of recommender system and its practical application. In addition, the long-standing gap between academic research and industrial applications related to recommender systems has not been well discussed, especially in the era of large language models. In this review, we introduce a novel taxonomy that originates from the intrinsic essence of recommendation, delving into the application of large language model-based recommendation systems and their industrial implementation. Specifically, we propose a three-tier structure that more accurately reflects the developmental progression of recommendation systems from research to practical implementation, including representing and understanding, scheming and utilizing, and industrial deployment. Furthermore, we discuss critical challenges and opportunities in this emerging field. A more up-to-date version of the papers is maintained at: https://github.com/jindongli-Ai/Next-Generation-LLM-based-Recommender-Systems-Survey.

Figures (9)

• Figure 1: An overview of the proposed three-tier taxonomy for LLM-based recommender systems: (a) representing and understanding, (b) scheming and utilizing, and (c) industrial deploying.
• Figure 2: Structure of this paper with representative works.
• Figure 3: The general pipeline of LLM-based recommender systems.
• Figure 4: Different paradigms for recommender systems: (a) Traditional Recommender System; (b) Non-Generative LLM-based Recommender System; and (c) Generative LLM-based Recommender System.
• Figure 5: Different paradigms for LLM-based recommender systems: (a) Naive Fine-Tuning: tailors the model to excel in specific recommendation tasks using domain/task-specific data; (b) Instruction Tuning2024_arXiv_Survey_instruction_tuning: optimizes the model's ability to follow diverse recommendation instructions and queries; (c) Low-Rank Adaptation (LoRA)2021_ICML_LoRA_low_rank: efficiently adjusts recommendation models with minimal changes, keeping the core parameters intact; (d) Direct Utilizing: utilizes the model directly in its pre-trained state without additional fine-tuning or training; (e) Prompt Tuning: optimizes specific recommendation prompts to tailor the model's responses, while keeping the underlying model parameters unchanged; and (f) In-Context Learning (ICL): uses contextual examples and cues within the input to guide the model's recommendations dynamically, without altering the model's parameters.

Theorems & Definitions (10)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Definition 6
• Definition 7
• Definition 8
• Definition 9
• Definition 10