Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boosting Knowledge Graph-based Recommendations through Confidence-Aware Augmentation with Large Language Models

Rui Cai, Chao Wang, Qianyi Cai, Dazhong Shen, Hui Xiong

TL;DR

This work tackles noise and maintenance challenges in knowledge-graph–driven recommendations by introducing CKG-LLMA, a framework that jointly leverages Large Language Models (LLMs) and knowledge graphs (KGs). It uses an LLM-based subgraph augmenter to enrich KGs under token constraints, a confidence-aware mixture-of-experts propagation to filter noisy edges, and a dual-view, two-step contrastive learning scheme to fuse KG and interaction data, along with a confidence-guided explanation generator. Empirical results on three public datasets show that CKG-LLMA consistently outperforms strong baselines in Recall@10 and NDCG@10, with ablations confirming the contribution of the confidence mechanism and the two-step augmentation. The approach yields robust, explainable recommendations and reduces risks of LLM hallucination by quantifying triplet confidence and leveraging cross-view stability, while keeping LLM usage offline for augmentation to control costs.

Abstract

Knowledge Graph-based recommendations have gained significant attention due to their ability to leverage rich semantic relationships. However, constructing and maintaining Knowledge Graphs (KGs) is resource-intensive, and the accuracy of KGs can suffer from noisy, outdated, or irrelevant triplets. Recent advancements in Large Language Models (LLMs) offer a promising way to improve the quality and relevance of KGs for recommendation tasks. Despite this, integrating LLMs into KG-based systems presents challenges, such as efficiently augmenting KGs, addressing hallucinations, and developing effective joint learning methods. In this paper, we propose the Confidence-aware KG-based Recommendation Framework with LLM Augmentation (CKG-LLMA), a novel framework that combines KGs and LLMs for recommendation task. The framework includes: (1) an LLM-based subgraph augmenter for enriching KGs with high-quality information, (2) a confidence-aware message propagation mechanism to filter noisy triplets, and (3) a dual-view contrastive learning method to integrate user-item interactions and KG data. Additionally, we employ a confidence-aware explanation generation process to guide LLMs in producing realistic explanations for recommendations. Finally, extensive experiments demonstrate the effectiveness of CKG-LLMA across multiple public datasets.

Boosting Knowledge Graph-based Recommendations through Confidence-Aware Augmentation with Large Language Models

TL;DR

This work tackles noise and maintenance challenges in knowledge-graph–driven recommendations by introducing CKG-LLMA, a framework that jointly leverages Large Language Models (LLMs) and knowledge graphs (KGs). It uses an LLM-based subgraph augmenter to enrich KGs under token constraints, a confidence-aware mixture-of-experts propagation to filter noisy edges, and a dual-view, two-step contrastive learning scheme to fuse KG and interaction data, along with a confidence-guided explanation generator. Empirical results on three public datasets show that CKG-LLMA consistently outperforms strong baselines in Recall@10 and NDCG@10, with ablations confirming the contribution of the confidence mechanism and the two-step augmentation. The approach yields robust, explainable recommendations and reduces risks of LLM hallucination by quantifying triplet confidence and leveraging cross-view stability, while keeping LLM usage offline for augmentation to control costs.

Abstract

Knowledge Graph-based recommendations have gained significant attention due to their ability to leverage rich semantic relationships. However, constructing and maintaining Knowledge Graphs (KGs) is resource-intensive, and the accuracy of KGs can suffer from noisy, outdated, or irrelevant triplets. Recent advancements in Large Language Models (LLMs) offer a promising way to improve the quality and relevance of KGs for recommendation tasks. Despite this, integrating LLMs into KG-based systems presents challenges, such as efficiently augmenting KGs, addressing hallucinations, and developing effective joint learning methods. In this paper, we propose the Confidence-aware KG-based Recommendation Framework with LLM Augmentation (CKG-LLMA), a novel framework that combines KGs and LLMs for recommendation task. The framework includes: (1) an LLM-based subgraph augmenter for enriching KGs with high-quality information, (2) a confidence-aware message propagation mechanism to filter noisy triplets, and (3) a dual-view contrastive learning method to integrate user-item interactions and KG data. Additionally, we employ a confidence-aware explanation generation process to guide LLMs in producing realistic explanations for recommendations. Finally, extensive experiments demonstrate the effectiveness of CKG-LLMA across multiple public datasets.

Paper Structure

This paper contains 30 sections, 17 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. KG-based Recommendations
  4. Unifying KGs and LLMs
  5. Methodology
  6. Definition and Overview
  7. LLM-based Subgraph Augmenter
  8. Subgraph Extraction and Prompt
  9. LLM Augmentation ($\mathcal{P}^{add}_{\mathcal{U}}$, $\mathcal{P}^{del}_{\mathcal{U}}$, $\mathcal{P}^{add}_{\mathcal{I}}$, $\mathcal{P}^{del}_{\mathcal{I}}$)
  10. Confidence-aware MOE Message Propogation
  11. Dual-view Two-step Contrastive Learning
  12. Step1: Differentiable Knowledge Graph Augmentation
  13. Step2: IG Enhancement via View-and-Stability
  14. Confidence-aware Contrastive Learning
  15. Model Optimization
  16. ...and 15 more sections

Figures (4)

  • Figure 1: The overview of the proposed CKG-LLMA framework for knowledge graph based recommendations.
  • Figure 2: Parameter Analysis of add ratio and delete ratio.
  • Figure 3: Prompts for LLM-based Subgraph Augmenter. Upper left is Item-view prompt and lower left is its related example. Upper right is User-view prompt and lower right is its related example.
  • Figure 4: The overall pipeline for Confidence-aware Explanation Generation. The left is the framework of generation process, and the right is the prompt template of LLM generation.