CADRL: Category-aware Dual-agent Reinforcement Learning for Explainable Recommendations over Knowledge Graphs

TL;DR

This work tackles the challenge of explainable KG-based recommendations by addressing two core issues: capturing contextual dependencies from neighboring information and efficiently exploring long recommendation paths. It introduces CADRL, a two-component approach consisting of CGGNN for high-order, context-aware item representations and a dual-agent reinforcement learning framework that collaboratively traverses long paths with a shared policy and a collaborative reward mechanism. The results on large-scale real-world datasets show that CADRL outperforms state-of-the-art baselines in both recommendation accuracy and efficiency, with extensive ablations confirming the contributions of CGGNN and DARL. The study highlights the practical value of combining category-level guidance with entity-level reasoning to produce explainable, scalable recommendations over knowledge graphs, and suggests future work integrating large language models to capture evolving user interests.

Abstract

Knowledge graphs (KGs) have been widely adopted to mitigate data sparsity and address cold-start issues in recommender systems. While existing KGs-based recommendation methods can predict user preferences and demands, they fall short in generating explicit recommendation paths and lack explainability. As a step beyond the above methods, recent advancements utilize reinforcement learning (RL) to find suitable items for a given user via explainable recommendation paths. However, the performance of these solutions is still limited by the following two points. (1) Lack of ability to capture contextual dependencies from neighboring information. (2) The excessive reliance on short recommendation paths due to efficiency concerns. To surmount these challenges, we propose a category-aware dual-agent reinforcement learning (CADRL) model for explainable recommendations over KGs. Specifically, our model comprises two components: (1) a category-aware gated graph neural network that jointly captures context-aware item representations from neighboring entities and categories, and (2) a dual-agent RL framework where two agents efficiently traverse long paths to search for suitable items. Finally, experimental results show that CADRL outperforms state-of-the-art models in terms of both effectiveness and efficiency on large-scale datasets.

Figures (7)

• Figure 1: A small fragment of a knowledge graph formed by the interactions of multiple users with the same shopping preferences. The red arrow is a 5-hop recommendation path starting from User 2 to a recommended item. To easily follow our work, we exemplify this fragment throughout the paper.
• Figure 2: Overview of CADRL. Different colored circles represent various types of entities, including items, brands, features, and users. After obtaining high-order representations of items from CGGNN, DARL fully interacts with the KG. Dual agents receive the state and reward as well as output the next action until the appropriate items are inferred.
• Figure 3: Effectiveness analysis of different modules in CGGNN on different datasets.
• Figure 4: Effectiveness analysis of different modules in DARL on different datasets.
• Figure 5: NDCG of the varying recommendation step $L$ for RL-based models.

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Definition 3
• Definition 4