Play to Your Strengths: Collaborative Intelligence of Conventional Recommender Models and Large Language Models

TL;DR

This paper addresses the gap that existing systems either rely on CRMs or LLMs for recommendations without leveraging their complementary strengths. It introduces CoReLLa, a framework that jointly trains a CRM and an LLM with an alignment loss, and uses entropy-based confidence to route hard samples to the LLM while easy samples are handled by the CRM. Through a three-stage training regimen and a layer-wise alignment objective, CoReLLa mitigates decision-boundary shifts and enables effective knowledge transfer between models. Empirical results on MovieLens-1M and Amazon-Books CTR tasks show that CoReLLa outperforms state-of-the-art CRM and LLM baselines, validating the practical benefit of data-aware collaboration between conventional and large-language recommender models.

Abstract

The rise of large language models (LLMs) has opened new opportunities in Recommender Systems (RSs) by enhancing user behavior modeling and content understanding. However, current approaches that integrate LLMs into RSs solely utilize either LLM or conventional recommender model (CRM) to generate final recommendations, without considering which data segments LLM or CRM excel in. To fill in this gap, we conduct experiments on MovieLens-1M and Amazon-Books datasets, and compare the performance of a representative CRM (DCNv2) and an LLM (LLaMA2-7B) on various groups of data samples. Our findings reveal that LLMs excel in data segments where CRMs exhibit lower confidence and precision, while samples where CRM excels are relatively challenging for LLM, requiring substantial training data and a long training time for comparable performance. This suggests potential synergies in the combination between LLM and CRM. Motivated by these insights, we propose Collaborative Recommendation with conventional Recommender and Large Language Model (dubbed \textit{CoReLLa}). In this framework, we first jointly train LLM and CRM and address the issue of decision boundary shifts through alignment loss. Then, the resource-efficient CRM, with a shorter inference time, handles simple and moderate samples, while LLM processes the small subset of challenging samples for CRM. Our experimental results demonstrate that CoReLLa outperforms state-of-the-art CRM and LLM methods significantly, underscoring its effectiveness in recommendation tasks.

Figures (2)

• Figure 1: Performance of conventional recommender model (CRM) and Large Language Model (LLM) on different groups.
• Figure 2: Framework of CoReLLa.