An Efficient LLM-based Evolutional Recommendation with Locate-Forget-Update Paradigm

TL;DR

Evolving user preferences in LLM-based recommender systems is challenged by costly full-model retraining and forgetting for inactive users when using fine-tuning. EvoRec introduces a Locate-Forget-Update paradigm that (a) localizes the most sensitive parameters responsible for preference drift, (b) uses a lightweight filtering model to discard outdated interactions, and (c) updates only the identified parameters with filtered data, achieving strong adaptation for active users while preserving inactive users’ preferences. The framework updates a small subset of parameters (about 30% of typical LoRA updates) and demonstrates superior efficiency and performance on two real-world datasets (Amazon Beauty and Toys) across multiple baselines and backbones. Its key contributions include precise parameter localization, effective data filtering, and an objective that balances alignment with new preferences and consistency for inactive users, enabling scalable, continual adaptation in dynamic recommendation settings. The results suggest EvoRec offers a practical, resource-efficient path for continuous evolution of LLM-based recommender systems in real-world deployment.

Abstract

Nowadays, Large Language Models (LLMs) have shown exceptional performance in sequential recommendations, and the adoption of LLM-based recommender systems (LLMRec) is becoming increasingly widespread in existing e-commerce platforms. Despite the impressive performance, the constant high volume of new user-item interactions makes it difficult to adapt to the evolution of user preference over time, especially for LLM-based recommender systems. The challenge arises from the large number of parameters in LLMs, which makes traditional evolution methods (i.e., Re-training or Fine-tuning) impractical. Specifically, Re-training with all interactions results in prohibitively high computational costs. On the other hand, fine-tuning with only new interactions leads to preference forgetting among inactive users, ultimately compromising overall performance. To tackle this problem, we propose EvoRec, an efficient Locate-Forget-Update framework designed for LLM-based recommender systems to model the evolution of user preferences. EvoRec identifies a small set of parameters associated with preference changes and updates them precisely, thereby saving computational resources while maintaining strong recommendation performance. Notably, the modified parameters account for only 30\% of LoRA adapter parameters, with no additional parameters introduced. Extensive experiments on two real-world datasets demonstrate that, compared to existing methods, EvoRec not only efficiently evolves LLMRec to adapt to the preferences of active users, but also preserves the interests of inactive users from being disturbed during evolution.

Figures (11)

• Figure 1: Illustration of preference drift challenge in personalized recommendation. (a) By applying efficient fine-tuning techniques to all user data before time T, LLMs achieve high recommendation performance at the current stage. (b) personalized recommendation systems face a fundamental challenge: user preferences are not static but evolve over time. As user preference drift (e.g., User A's preference changes from clothing to electronics), existing LLM-based recommenders struggle to adapt and provide accurate personalization. In contrast, for users whose preferences remain stable (e.g., Users B and C), the models can still deliver reliable recommendations.
• Figure 2: Limitations of two common evolution paradigms: (a) Re-training with new data preserves performance for both evolving (user A) and stable (users B, C) preferences but incurs high cost; (b) Finetuning adapts to user A but causes preference forgetting for users B, C; (c) Results of three consecutive finetuning rounds further confirm this preference forgetting phenomenon.
• Figure 3: Illustration of our proposed EvoRec framework. First, we input both the historical and recent interaction sequences of users into the outdated LLM-Based Recommender System. By tracking the differences in their hidden states, we identify the sensitive parameters that exhibit the greatest divergence between past and updated preferences. Second, we employ a filtering model to filter out outdated interactions based on relevance scores. Finally, EvoRec updates the identified parameters using the filtered data, ensuring adaptation to users' latest preferences.
• Figure 4: Illustration of the outdated interaction removal process. (a) Fine-tuning the filtering model with recent data to quickly adapt to users’ latest preferences. (b) Removing interactions with low contribution scores based on their impact on user sequence modeling (considered outdated). (c) Example after removing outdated interactions.
• Figure 5: Radar chart illustrating the performance of different incremental learning paradigms on $\mathcal{U}_A$, $\mathcal{U}_I$, and $\mathcal{U}$ after a single update (using TALLRec as the recommendation method).