CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured Knowledge Aggregation

TL;DR

A novel context-aware graph-attention model (Context-aware GAT) is proposed, designed to effectively assimilate global features from relevant knowledge graphs through a context-enhanced knowledge aggregation mechanism, which outperforms conventional GNN-based language models in terms of performance.

Abstract

Commonsense knowledge is crucial to many natural language processing tasks. Existing works usually incorporate graph knowledge with conventional graph neural networks (GNNs), resulting in a sequential pipeline that compartmentalizes the encoding processes for textual and graph-based knowledge. This compartmentalization does, however, not fully exploit the contextual interplay between these two types of input knowledge. In this paper, a novel context-aware graph-attention model (Context-aware GAT) is proposed, designed to effectively assimilate global features from relevant knowledge graphs through a context-enhanced knowledge aggregation mechanism. Specifically, the proposed framework employs an innovative approach to representation learning that harmonizes heterogeneous features by amalgamating flattened graph knowledge with text data. The hierarchical application of graph knowledge aggregation within connected subgraphs, complemented by contextual information, to bolster the generation of commonsense-driven dialogues is analyzed. Empirical results demonstrate that our framework outperforms conventional GNN-based language models in terms of performance. Both, automated and human evaluations affirm the significant performance enhancements achieved by our proposed model over the concept flow baseline.

Figures (6)

• Figure 1: Illustration of the proposed model with an example. The retrieved facts are fed to the graph model, then the model learns the representations of concepts by aggregating the knowledge layer by layer. Finally, responses are generated with these aggregated features.
• Figure 2: Overview of the proposed model.
• Figure 3: The Context-aware GAT firstly transforms knowledge from facts into numeric vectors (in yellow). Through feature forwarding, the root nodes of each graph attentively read and aggregate all knowledge and become higher-level representations (from yellow to green, and then green to red).
• Figure 4: The learned attention probability density curves on knowledge facts.
• Figure 5: A box plot to analyse attention scores learned by context-aware GAT to aggregate features from one-hop and two-hop knowledge. Overall: average attention of all knowledge; Golden: average attention of golden knowledge; Output: average attention of knowledge in generated responses.