Large Language Models Enhanced Collaborative Filtering

TL;DR

The paper presents LLM-CF, a framework that distills world knowledge and reasoning from Large Language Models into collaborative filtering for recommender systems. It splits the workflow into offline instruction-tuning (RecGen-LLaMA) to generate CoT reasoning and an online ICT module that retrieves in-context CoT examples to produce CF-augmented features for backbone RSs, all while keeping LLM inference decoupled from online service. Empirical results across six backbone models and three Amazon datasets show significant gains in ranking and retrieval tasks, outperforming KD and KAR baselines and validating the efficacy of in-context CoT-guided CF features. The approach offers a scalable, efficient path to leverage LLM capabilities for CF without incurring real-time LLM latency in production.

Abstract

Recent advancements in Large Language Models (LLMs) have attracted considerable interest among researchers to leverage these models to enhance Recommender Systems (RSs). Existing work predominantly utilizes LLMs to generate knowledge-rich texts or utilizes LLM-derived embeddings as features to improve RSs. Although the extensive world knowledge embedded in LLMs generally benefits RSs, the application can only take limited number of users and items as inputs, without adequately exploiting collaborative filtering information. Considering its crucial role in RSs, one key challenge in enhancing RSs with LLMs lies in providing better collaborative filtering information through LLMs. In this paper, drawing inspiration from the in-context learning and chain of thought reasoning in LLMs, we propose the Large Language Models enhanced Collaborative Filtering (LLM-CF) framework, which distils the world knowledge and reasoning capabilities of LLMs into collaborative filtering. We also explored a concise and efficient instruction-tuning method, which improves the recommendation capabilities of LLMs while preserving their general functionalities (e.g., not decreasing on the LLM benchmark). Comprehensive experiments on three real-world datasets demonstrate that LLM-CF significantly enhances several backbone recommendation models and consistently outperforms competitive baselines, showcasing its effectiveness in distilling the world knowledge and reasoning capabilities of LLM into collaborative filtering.

Figures (5)

• Figure 1: LLM-CF integrates LLM-based world knowledge and reasoning with collaborative filtering to improve recommendation performance, using LLMs with recommender capability and decoupled latency-free offline generation.
• Figure 2: Example of the textual format recommendation features and Instruction-Tuning prompt.
• Figure 3: The overall architecture of the proposed model LLM-CF. (1) CoT Reasoning Generation: Utilize Chain of Thought Prompt to generate CoT reasoning based on RecGen-LLaMA. (2) In-context CoT Examples Retrieval: Identify top-K similar historical recommendation data with current recommendation data to form in-context CoT examples. (3) In-context Chain of Thought (ICT) Module: Employ transformer decoder layers to learn world-knowledge and reasoning guided CF feature.
• Figure 4: The recommendation and general capabilities of LLaMA2 after fine-tuning on different recommendation data using various methods.
• Figure 5: The recommendation performance of LLM-CF on 6 backbone models across three real-world datasets as it varies with the change in In-context CoT examples length and positive/negative example constraints. The red dots indicate the best-performing setting for the current backbone model and dataset.