Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation

TL;DR

The paper addresses the problem of dull open-domain dialogue responses by introducing a content introducing framework that first predicts a noun keyword via pointwise mutual information and then builds a reply around that keyword with a backward and forward sequence model (seq2BF). This approach decouples content selection from fluent generation and allows the keyword to appear at any position in the reply. Empirical results on a large Chinese dataset show that keyword-guided seq2BF outperforms standard seq2seq in human evaluation and increases entropy, indicating richer, more informative outputs. The method offers a practical strategy to inject salient content into generative dialogue systems and can be extended to other content-rich generation tasks such as generative question answering.

Abstract

Using neural networks to generate replies in human-computer dialogue systems is attracting increasing attention over the past few years. However, the performance is not satisfactory: the neural network tends to generate safe, universally relevant replies which carry little meaning. In this paper, we propose a content-introducing approach to neural network-based generative dialogue systems. We first use pointwise mutual information (PMI) to predict a noun as a keyword, reflecting the main gist of the reply. We then propose seq2BF, a "sequence to backward and forward sequences" model, which generates a reply containing the given keyword. Experimental results show that our approach significantly outperforms traditional sequence-to-sequence models in terms of human evaluation and the entropy measure, and that the predicted keyword can appear at an appropriate position in the reply.

Figures (1)

• Figure 1: An overview of our content-introducing approach to generative dialogue systems.