Knowledge Graphs and Pre-trained Language Models enhanced Representation Learning for Conversational Recommender Systems

TL;DR

This paper presents Knowledge-Enhanced Entity Representation Learning (KERL) for conversational recommender systems, addressing the limits of relying solely on KG topology by encoding entity textual descriptions with a pre-trained language model and enriching them with KG structure. It introduces a knowledge graph encoding module, a knowledge-enhanced recommendation module with a conversational entity encoder, history encoder, and contrastive alignment, and a knowledge-enhanced BART-based response generation module. A high-quality WikiMKG knowledge graph with aligned entity descriptions is built to support the study. Empirical results on two movie CRS benchmarks (ReDial and INSPIRED) show state-of-the-art performance for both recommendation and response generation, with ablations highlighting the importance of entity descriptions, temporal encoding, and contrastive learning. The findings demonstrate the practical impact of integrating semantic entity information with KG structure for more accurate recommendations and informative, fluent responses in CRS.

Abstract

Conversational recommender systems (CRS) utilize natural language interactions and dialogue history to infer user preferences and provide accurate recommendations. Due to the limited conversation context and background knowledge, existing CRSs rely on external sources such as knowledge graphs to enrich the context and model entities based on their inter-relations. However, these methods ignore the rich intrinsic information within entities. To address this, we introduce the Knowledge-Enhanced Entity Representation Learning (KERL) framework, which leverages both the knowledge graph and a pre-trained language model to improve the semantic understanding of entities for CRS. In our KERL framework, entity textual descriptions are encoded via a pre-trained language model, while a knowledge graph helps reinforce the representation of these entities. We also employ positional encoding to effectively capture the temporal information of entities in a conversation. The enhanced entity representation is then used to develop a recommender component that fuses both entity and contextual representations for more informed recommendations, as well as a dialogue component that generates informative entity-related information in the response text. A high-quality knowledge graph with aligned entity descriptions is constructed to facilitate our study, namely the Wiki Movie Knowledge Graph (WikiMKG). The experimental results show that KERL achieves state-of-the-art results in both recommendation and response generation tasks.

Figures (5)

• Figure 1: Example of a KG that incorporates entity descriptions. The figure suggests that descriptions contain rich information and can help improve the semantic understanding of entities
• Figure 2: The overview of the framework of the proposed KERL in a movie recommendation scenario. The Attention Network (AN) selectively focuses on relevant tokens. Positional Encoding (PE) and Self-Attention (SA) mechanisms preserve the sequence order and context, respectively. Context Cross-Attention (CA) and KG Cross-Attention integrate conversational and knowledge graph cues. The Recommendation Module matches items to user preferences, and the Response Generation Module formulates natural language suggestions.
• Figure 3: The Knowledge Graph Encoding Module employs a PLM for textual semantics and a GNN for structural relations. It generates entity embeddings that include item embeddings, which are a subset used for recommendations. "AN" denotes the attention network.
• Figure 4: Comparison of different pooling methods on the ReDial and INSPIRED datasets.
• Figure 5: Performance comparison of different KERL variants on recommendation tasks on the ReDial Dataset. D, KE and CL refer to entity description, knowledge embedding method and contrastive learning.