Real-Time Personalization for LLM-based Recommendation with Customized In-Context Learning

TL;DR

RecICL is proposed, which customizes recommendation-specific in-context learning for real-time recommendations for real-time recommendations, and organizes training examples in an in-context learning format, ensuring that in-context learning ability is preserved and aligned with the recommendation task during tuning.

Abstract

Frequently updating Large Language Model (LLM)-based recommender systems to adapt to new user interests -- as done for traditional ones -- is impractical due to high training costs, even with acceleration methods. This work explores adapting to dynamic user interests without any model updates by leveraging In-Context Learning (ICL), which allows LLMs to learn new tasks from few-shot examples provided in the input. Using new-interest examples as the ICL few-shot examples, LLMs may learn real-time interest directly, avoiding the need for model updates. However, existing LLM-based recommenders often lose the in-context learning ability during recommendation tuning, while the original LLM's in-context learning lacks recommendation-specific focus. To address this, we propose RecICL, which customizes recommendation-specific in-context learning for real-time recommendations. RecICL organizes training examples in an in-context learning format, ensuring that in-context learning ability is preserved and aligned with the recommendation task during tuning. Extensive experiments demonstrate RecICL's effectiveness in delivering real-time recommendations without requiring model updates. Our code is available at https://github.com/ym689/rec_icl.

Figures (7)

• Figure 1: A comparative analysis of TALLRec, BinLLM, and HashGNN models using both Amazon-Book and Amazon-Movie datasets for performance assessment. A higher "PDM" indicates a greater benefit of updating the mode. A higher "PDT" signifies a more significant impact of shifts in user interests on the model's performance.
• Figure 2: Performance of TALLRec, BinLLM, and General LLM in scenarios with and without in-context learning on each dataset. The x-axis shows the name of the method, while the y-axis shows the performance improvement brought by using in-context learning, where in-context learning refers to selecting the four most recent interactions for each user to formulate the few-shot examples.
• Figure 3: Overview of our RecICL pipeline, primarily consists of three stages: Sample Construction, Model Training, and Real-time Inference. Here we define the few-shot number as 4.
• Figure 4: The performance of the model on different test sets after training on $\mathcal{D}_{train}$. The x-axis represents practical data partitions, with larger subscripts indicating a greater shift in user interests compared to the training set. The y-axis shows the corresponding AUC metric for each data partition.
• Figure 5: The performance of RecICL trained with varying numbers of few-shot samples (left) and its inference overhead on the entire test set (right). Note that when the number of few-shot samples is 0, it is equivalent to TALLRec.