RecCocktail: A Generalizable and Efficient Framework for LLM-Based Recommendation

TL;DR

This paper tackles the challenge of building generalizable yet domain-adaptive LLM-based recommender systems. It introduces RecCocktail, a three-step framework that learns a domain-general LoRA (base spirit) from multi-domain data and a domain-specific LoRA (ingredient) for a target domain, which are then merged in weight space via an entropy-guided LoRA cocktail. The method preserves a single LoRA at inference, enabling plug-and-play adaptation without extra computational cost, and uses test-time entropy minimization to adapt merging coefficients to the data distribution. Extensive experiments across warm-start and cold-start settings on multiple datasets show RecCocktail achieves state-of-the-art performance, with Ablation and analysis supporting the effectiveness of adaptive merging and the benefits of combining general and domain-specific knowledge for robust recommendations.

Abstract

Large Language Models (LLMs) have achieved remarkable success in recent years, owing to their impressive generalization capabilities and rich world knowledge. To capitalize on the potential of using LLMs as recommender systems, mainstream approaches typically focus on two paradigms. The first paradigm designs multi-domain or multi-task instruction data for generalizable recommendation, so as to align LLMs with general recommendation areas and deal with cold-start recommendation. The second paradigm focuses on enhancing domain-specific recommendation tasks, improving performance in warm recommendation scenarios. While most previous works treat these two paradigms separately, we argue that they have complementary advantages, and combining them can yield better results. In this paper, we propose a generalizable and efficient LLM-based recommendation framework RecCocktail. Our approach begins with fine-tuning a "base spirit" LoRA module using domain-general recommendation instruction data to align LLM with recommendation knowledge. Next, given users' behavior of a specific domain, we construct a domain-specific "ingredient" LoRA module. We then provide an entropy-guided adaptive merging method to mix the "base spirit" and the "ingredient" in the weight space. Please note that, RecCocktail combines the advantages of the existing two paradigms without introducing additional time or space overhead during the inference phase. Moreover, RecCocktail is efficient with plug and play, as the "base spirit" LoRA is trained only once, and any domain-specific "ingredient" can be efficiently mixed with only domain-specific fine-tuning. Extensive experiments on multiple datasets under both warm and cold-start recommendation scenarios validate the effectiveness and generality of the proposed RecCocktail.

Figures (5)

• Figure 1: Illustration of different LLM-based recommendation paradigms. (1) Breadth-oriented paradigm. (2) Depth-oriented paradigm. (3) Our proposed RecCocktail.
• Figure 2: Illustration of our proposed RecCocktail framework.
• Figure 3: Illustration of task arithmetic ilharco2023editing. (a) Task vectors are obtained by subtracting pre-trained weights from fine-tuned weights. (b) Adding task vectors improves performance on multiple tasks.
• Figure 4: (a) and (b) are the coefficients $\lambda_1$ and $\lambda_2$ calculated by entropy-guided adaptive merging. (c) and (d) are the impact of the number of unlabeled test data.
• Figure 5: Case study of RecCocktail and GPT-4 explainable recommendation results.