Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LSTM based Conversation Models

Yi Luan, Yangfeng Ji, Mari Ostendorf

TL;DR

This work presents an LSTM-based framework for two-party conversations that jointly models local turn context, global topic context via LDA, and speaker roles. It introduces three model variants—R-Conv, LDA-Conv, and the combined R-LDA-Conv—where role information biases the output and topic context informs predictions. Evaluations on the Ubuntu Dialogue Corpus show that incorporating role and topic context yields substantial improvements in perplexity and response ranking, with R-LDA-Conv performing best. Qualitative analyses reveal clear role-specific generation patterns, supporting the practical value of role-aware dialogue systems in goal-directed settings.

Abstract

In this paper, we present a conversational model that incorporates both context and participant role for two-party conversations. Different architectures are explored for integrating participant role and context information into a Long Short-term Memory (LSTM) language model. The conversational model can function as a language model or a language generation model. Experiments on the Ubuntu Dialog Corpus show that our model can capture multiple turn interaction between participants. The proposed method outperforms a traditional LSTM model as measured by language model perplexity and response ranking. Generated responses show characteristic differences between the two participant roles.

LSTM based Conversation Models

TL;DR

This work presents an LSTM-based framework for two-party conversations that jointly models local turn context, global topic context via LDA, and speaker roles. It introduces three model variants—R-Conv, LDA-Conv, and the combined R-LDA-Conv—where role information biases the output and topic context informs predictions. Evaluations on the Ubuntu Dialogue Corpus show that incorporating role and topic context yields substantial improvements in perplexity and response ranking, with R-LDA-Conv performing best. Qualitative analyses reveal clear role-specific generation patterns, supporting the practical value of role-aware dialogue systems in goal-directed settings.

Abstract

In this paper, we present a conversational model that incorporates both context and participant role for two-party conversations. Different architectures are explored for integrating participant role and context information into a Long Short-term Memory (LSTM) language model. The conversational model can function as a language model or a language generation model. Experiments on the Ubuntu Dialog Corpus show that our model can capture multiple turn interaction between participants. The proposed method outperforms a traditional LSTM model as measured by language model perplexity and response ranking. Generated responses show characteristic differences between the two participant roles.

Paper Structure

This paper contains 14 sections, 10 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model
  4. Recurrent Neural Network Language Models
  5. Conversation Models with Speaker Roles
  6. Incorporating global topic context
  7. Experiments
  8. Experimental setup
  9. Evaluation Metrics
  10. Quantative Evaluation
  11. Perplexity
  12. Response ranking
  13. Qualitative Analysis
  14. Summary

Figures (1)

  • Figure 1: The R-LDA-Conv model. The turn-level LDA feature $\boldsymbol{s}_{t}$ is concatenated with word-level hidden layer $\boldsymbol{h}_{t,i}$ and the output weight matrix $\mathbf{W}_{r_t}$ is role specific.