Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation

TL;DR

The paper tackles hallucinations and knowledge gaps in LLM-based recommender systems by introducing K-RagRec, a knowledge-graph retrieval-augmented framework. It constructs semantic-hop sub-graphs from an external KG, employs a popularity-based retrieval policy, and encodes retrieved sub-graphs with GNNs before integrating them as soft prompts to frozen LLMs. Key innovations include Hop-Field semantic indexing, Top-N sub-graph re-ranking, and a lightweight training regime that updates only graph components and a projector. Experiments on MovieLens and Amazon Book across multiple backbones demonstrate substantial gains in accuracy and recall, with favorable efficiency and notable zero-shot generalization, validating the practicality of structure-aware KG-RAG for real-world recommendations.

Abstract

Recommender systems have become increasingly vital in our daily lives, helping to alleviate the problem of information overload across various user-oriented online services. The emergence of Large Language Models (LLMs) has yielded remarkable achievements, demonstrating their potential for the development of next-generation recommender systems. Despite these advancements, LLM-based recommender systems face inherent limitations stemming from their LLM backbones, particularly issues of hallucinations and the lack of up-to-date and domain-specific knowledge. Recently, Retrieval-Augmented Generation (RAG) has garnered significant attention for addressing these limitations by leveraging external knowledge sources to enhance the understanding and generation of LLMs. However, vanilla RAG methods often introduce noise and neglect structural relationships in knowledge, limiting their effectiveness in LLM-based recommendations. To address these limitations, we propose to retrieve high-quality and up-to-date structure information from the knowledge graph (KG) to augment recommendations. Specifically, our approach develops a retrieval-augmented framework, termed K-RagRec, that facilitates the recommendation generation process by incorporating structure information from the external KG. Extensive experiments have been conducted to demonstrate the effectiveness of our proposed method.

Figures (6)

• Figure 1: Illustration of the issues of hallucinations and lack of domain-specific knowledge in LLM-based recommender systems and how they can be addressed by knowledge graph retrieval-augmented generation (KG RAG).
• Figure 2: The overview of the K-RagRec. It contains five key components: Hop-Field Knowledge Sub-graphs for Semantic Indexing, Popularity Selective Retrieval Policy, Knowledge Sub-graphs Retrieval, Knowledge Sub-graphs Re-Ranking, and Knowledge-augmented Recommendation.
• Figure 3: Comparison among K-RagRec and its four ablated variants on MovieLens-1M and Amazon Book datasets and LLama-2-7b across metrics Accuracy, Recall@3 and Recall@5.
• Figure 4: Effect of popularity selective retrieval policy threshold $p$ on MovieLens-1M and LLama-2-7b across metrics Accuracy, Recall@5 and inference time (seconds) for K-RagRec.
• Figure 5: Effect of retrieved knowledge sub-graph numbers $K$ on Amazon Book datasets and LLama-2-7b across metrics Accuracy, Recall@3 and Recall@5 for K-RagRec.