Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models

TL;DR

This work tackles the open-world knowledge gap in recommender systems by introducing KAR, a model-agnostic framework that retrieves reasoning about user preferences and factual item knowledge from large language models. By using factorization prompting and a hybrid-expert adaptor, KAR converts textual knowledge into dense, compact vectors that can be seamlessly integrated into existing recommendation models, with offline prestorage to meet latency constraints. Extensive experiments on MovieLens-1M and Amazon-Books demonstrate consistent gains over strong baselines, and deployment on Huawei platforms shows meaningful online improvements. The approach highlights the practical viability of leveraging LLM-derived open-world knowledge to boost real-world recommendation performance while addressing reliability and efficiency concerns.

Abstract

Recommender systems play a vital role in various online services. However, the insulated nature of training and deploying separately within a specific domain limits their access to open-world knowledge. Recently, the emergence of large language models (LLMs) has shown promise in bridging this gap by encoding extensive world knowledge and demonstrating reasoning capability. Nevertheless, previous attempts to directly use LLMs as recommenders have not achieved satisfactory results. In this work, we propose an Open-World Knowledge Augmented Recommendation Framework with Large Language Models, dubbed KAR, to acquire two types of external knowledge from LLMs -- the reasoning knowledge on user preferences and the factual knowledge on items. We introduce factorization prompting to elicit accurate reasoning on user preferences. The generated reasoning and factual knowledge are effectively transformed and condensed into augmented vectors by a hybrid-expert adaptor in order to be compatible with the recommendation task. The obtained vectors can then be directly used to enhance the performance of any recommendation model. We also ensure efficient inference by preprocessing and prestoring the knowledge from the LLM. Extensive experiments show that KAR significantly outperforms the state-of-the-art baselines and is compatible with a wide range of recommendation algorithms. We deploy KAR to Huawei's news and music recommendation platforms and gain a 7\% and 1.7\% improvement in the online A/B test, respectively.

Figures (5)

• Figure 1: Comparison between (a) closed recommender systems and (b) open-world recommender systems.
• Figure 2: The overall framework of KAR, consisting of knowledge reasoning and generation stage, knowledge adaptation stage, and knowledge utilization stage. Knowledge reasoning and generation stage leverages our designed factorization prompting to extract the reasoning and factual knowledge from LLMs. Knowledge adaptation stage converts textual open-world knowledge into compact and the reasoning and fact augmented representations suitable for recommendation. Knowledge utilization stage integrates the reasoning and fact augmented vectors into an existing recommendation model.
• Figure 3: Example prompts for KAR. The green, purple, and yellow text bubbles represent the prompt template, the content to be filled in the template, and the response generated by LLMs, respectively (some text has been omitted due to the page limits).
• Figure 4: Comparison between knowledge from knowledge graph and LLM on MovieLens-1M dataset.
• Figure 5: Ablation study about reasoning and factual knowledge on Amazon-Books dataset.