Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems

Yiping Song, Rui Yan, Xiang Li, Dongyan Zhao, Ming Zhang

TL;DR

This work tackles open-domain dialog by integrating retrieval and generation into a single ensemble. A retrieval system first extracts a best-matching reply $r^*$ for a query $q$, and a biseq2seq generator uses both $q$ and $r^*$ to produce a new candidate $r^+$, which is then post-reranked against $r^*$ to deliver the final response. The key contributions are the introduction of the biseq2seq model, the post-reranking mechanism, and the demonstration that the ensemble consistently surpasses its individual components in both human judgments and BLEU-based metrics, with substantial gains in user-sensible, substance-rich replies. This approach advances practical open-domain chat by leveraging complementary strengths of retrieval and neural generation, offering a robust path to higher-quality, more engaging conversational systems.

Abstract

Open-domain human-computer conversation has attracted much attention in the field of NLP. Contrary to rule- or template-based domain-specific dialog systems, open-domain conversation usually requires data-driven approaches, which can be roughly divided into two categories: retrieval-based and generation-based systems. Retrieval systems search a user-issued utterance (called a query) in a large database, and return a reply that best matches the query. Generative approaches, typically based on recurrent neural networks (RNNs), can synthesize new replies, but they suffer from the problem of generating short, meaningless utterances. In this paper, we propose a novel ensemble of retrieval-based and generation-based dialog systems in the open domain. In our approach, the retrieved candidate, in addition to the original query, is fed to an RNN-based reply generator, so that the neural model is aware of more information. The generated reply is then fed back as a new candidate for post-reranking. Experimental results show that such ensemble outperforms each single part of it by a large margin.

Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems

TL;DR

This work tackles open-domain dialog by integrating retrieval and generation into a single ensemble. A retrieval system first extracts a best-matching reply for a query , and a biseq2seq generator uses both and to produce a new candidate , which is then post-reranked against to deliver the final response. The key contributions are the introduction of the biseq2seq model, the post-reranking mechanism, and the demonstration that the ensemble consistently surpasses its individual components in both human judgments and BLEU-based metrics, with substantial gains in user-sensible, substance-rich replies. This approach advances practical open-domain chat by leveraging complementary strengths of retrieval and neural generation, offering a robust path to higher-quality, more engaging conversational systems.

Abstract

Open-domain human-computer conversation has attracted much attention in the field of NLP. Contrary to rule- or template-based domain-specific dialog systems, open-domain conversation usually requires data-driven approaches, which can be roughly divided into two categories: retrieval-based and generation-based systems. Retrieval systems search a user-issued utterance (called a query) in a large database, and return a reply that best matches the query. Generative approaches, typically based on recurrent neural networks (RNNs), can synthesize new replies, but they suffer from the problem of generating short, meaningless utterances. In this paper, we propose a novel ensemble of retrieval-based and generation-based dialog systems in the open domain. In our approach, the retrieved candidate, in addition to the original query, is fed to an RNN-based reply generator, so that the neural model is aware of more information. The generated reply is then fed back as a new candidate for post-reranking. Experimental results show that such ensemble outperforms each single part of it by a large margin.

Paper Structure

This paper contains 15 sections, 4 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. The Proposed Model Ensemble
  3. Overview
  4. Retrieval-Based Dialog System
  5. The biseq2seq Utterance Generator
  6. Post-Reranking
  7. Training
  8. Evaluation
  9. Experimental Setup
  10. Competing Methods
  11. Overall Performance
  12. Analysis and Discussion
  13. Case Study
  14. Related Work
  15. Conclusion and Future Work

Figures (3)

  • Figure 1: The overall architecture of our model ensemble. We combine retrieval and generative dialog systems by ① enhancing the generator with the retrieved candidate and by ② post-reranking of both retrieved and generated candidates.
  • Figure 2: The biseq2seq model, which takes as input a query $q$ and a retrieved candidate reply $r^*$; it outputs a new reply $r^+$.
  • Figure 3: The percentage by which our post-reranker chooses a retrieved reply or a generated reply. (a) Ensemble of Retrieval and seq2seq; (b) Ensemble of Retrieval and biseq2seq.