Collaborative Knowledge Fusion: A Novel Approach for Multi-task Recommender Systems via LLMs

TL;DR

This work introduces a novel framework known as CKF, specifically developed to boost multi-task recommendations via personalized collaborative knowledge fusion into LLMs, and develops Multi-Lora, a new parameter-efficient approach for multi-task optimization, adept at distinctly segregating task-shared and task-specific information.

Abstract

Owing to the impressive general intelligence of large language models (LLMs), there has been a growing trend to integrate them into recommender systems to gain a more profound insight into human interests and intentions. Existing LLMs-based recommender systems primarily leverage item attributes and user interaction histories in textual format, improving the single task like rating prediction or explainable recommendation. Nevertheless, these approaches overlook the crucial contribution of traditional collaborative signals in discerning users' profound intentions and disregard the interrelatedness among tasks. To address these limitations, we introduce a novel framework known as CKF, specifically developed to boost multi-task recommendations via personalized collaborative knowledge fusion into LLMs. Specifically, our method synergizes traditional collaborative filtering models to produce collaborative embeddings, subsequently employing the meta-network to construct personalized mapping bridges tailored for each user. Upon mapped, the embeddings are incorporated into meticulously designed prompt templates and then fed into an advanced LLM to represent user interests. To investigate the intrinsic relationship among diverse recommendation tasks, we develop Multi-Lora, a new parameter-efficient approach for multi-task optimization, adept at distinctly segregating task-shared and task-specific information. This method forges a connection between LLMs and recommendation scenarios, while simultaneously enriching the supervisory signal through mutual knowledge transfer among various tasks. Extensive experiments and in-depth robustness analyses across four common recommendation tasks on four large public data sets substantiate the effectiveness and superiority of our framework.

Figures (28)

• Figure 1: The combination of collaborative knowledge and LLMs. Lines in distinct colors depict various user-item interactions, while icons of fire and snow symbolize trainable and frozen LLM parameters, respectively.
• Figure 2: Difference in related work. $m$ denotes the generalized mapping function and $m_{i}$ refers to the personalized mapping function. The comparison reveals that the majority of LLM-based research focuses on the individual task without incorporating joint optimization across multiple tasks. CoLLM's collaborative knowledge mapping ignores the semantic gap and personalized nature of users and items.
• Figure 3: Overview of the framework, where orange, blue, and green backgrounds represent CKEM, KFM, and MTM parts respectively. Initially, user and item embeddings are generated via the collaborative filtering model, constituting what is referred to as collaborative knowledge. Subsequently, historical embeddings are utilized as input to two meta-networks, yielding personalized mapping functions for both user and candidate item. These mapping functions facilitate the transmission of collaborative knowledge to the LLM, in conjunction with prompt templates for multi-task recommendation. The whole optimization process is carried out using the designed Multi-Lora strategy.
• Figure 4: Task Prompt. Black fonts are input content, red fonts are task instructions. The two unk are placeholders for the user and item collaboration vectors respectively.
• Figure 5: Warm-cold scenarios (Movie-Lens data set). We select the strong baselines MF, TALLRec, CKF-S, CoLLM, and CKF for comparison. The Movie-Lens data set lacks comment data, precluding comparisons of explainable recommendations.