CherryRec: Enhancing News Recommendation Quality via LLM-driven Framework
Shaohuang Wang, Lun Wang, Yunhan Bu, Tianwei Huang
TL;DR
This work tackles real-time news recommendation using Large Language Models by addressing slow inference and dependence on pre-trained knowledge. It proposes CherryRec, a three-module framework comprising KnRS for rapid candidate filtering, CnLE for user-preference-aware evaluation via a fine-tuned LLM, and VaNS for synthesizing a final personalized score, formalized through $V(u,N)=\!w_1 f_1(u,N)+w_2 f_2(N)+w_3 f_3(N)+w_4 f_4(H)+w_5 f_5(N)$ and a final ranking mechanism. Empirical results on MIND, Yahoo R6B, and Adressa show consistent improvements in MRR@5, NDCG@5, and Recall@5 over strong baselines (including SASRec, NARM, BERT4Rec, P5, GPT4Rec, and TALLRec), with notable gains in fast retrieval scenarios and robust performance across datasets. The study also demonstrates the importance of each module via ablation and highlights CherryRec’s ability to deliver high-quality, timely recommendations while maintaining efficiency, suggesting practical impact for production news feeds and adaptable, multi-task tuning. Overall, CherryRec offers a concrete, scalable approach to leveraging LLMs for dynamic, user-aware news recommendation, with potential extensions to multimodal content and broader domains.
Abstract
Large Language Models (LLMs) have achieved remarkable progress in language understanding and generation. Custom LLMs leveraging textual features have been applied to recommendation systems, demonstrating improvements across various recommendation scenarios. However, most existing methods perform untrained recommendation based on pre-trained knowledge (e.g., movie recommendation), and the auto-regressive generation of LLMs leads to slow inference speeds, making them less effective in real-time recommendations.To address this, we propose a framework for news recommendation using LLMs, named \textit{CherryRec}, which ensures the quality of recommendations while accelerating the recommendation process. Specifically, we employ a Knowledge-aware News Rapid Selector to retrieve candidate options based on the user's interaction history. The history and retrieved items are then input as text into a fine-tuned LLM, the Content-aware News Llm Evaluator, designed to enhance news recommendation capabilities. Finally, the Value-aware News Scorer integrates the scores to compute the CherryRec Score, which serves as the basis for the final recommendation.We validate the effectiveness of the proposed framework by comparing it with state-of-the-art baseline methods on benchmark datasets. Our experimental results consistently show that CherryRec outperforms the baselines in both recommendation performance and efficiency.The project resource can be accessed at: \url{https://github.com/xxxxxx}