Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unsupervised Flow Discovery from Task-oriented Dialogues

Patrícia Ferreira, Daniel Martins, Ana Alves, Catarina Silva, Hugo Gonçalo Oliveira

TL;DR

The paper addresses the high manual cost of designing task-oriented dialogue flows by proposing an unsupervised pipeline that embeds utterances, clusters them into semantic states, and builds a transition graph with Start/End nodes. States are labelled automatically to yield human-interpretable flows, which are visualized and evaluated using a novel automatic metric on a well-known TOD dataset, MultiWOZ 2.2. The key contributions are the unsupervised flow discovery method, the graph-based representation of dialogue dynamics, and a validation approach demonstrating substantial predictability of test transitions (over 80% at a lower threshold) without annotated data. This work enables scalable flow extraction from real dialogue histories, aiding rapid design, auditing, and explainability of TOD systems across domains.

Abstract

The design of dialogue flows is a critical but time-consuming task when developing task-oriented dialogue (TOD) systems. We propose an approach for the unsupervised discovery of flows from dialogue history, thus making the process applicable to any domain for which such an history is available. Briefly, utterances are represented in a vector space and clustered according to their semantic similarity. Clusters, which can be seen as dialogue states, are then used as the vertices of a transition graph for representing the flows visually. We present concrete examples of flows, discovered from MultiWOZ, a public TOD dataset. We further elaborate on their significance and relevance for the underlying conversations and introduce an automatic validation metric for their assessment. Experimental results demonstrate the potential of the proposed approach for extracting meaningful flows from task-oriented conversations.

Unsupervised Flow Discovery from Task-oriented Dialogues

TL;DR

The paper addresses the high manual cost of designing task-oriented dialogue flows by proposing an unsupervised pipeline that embeds utterances, clusters them into semantic states, and builds a transition graph with Start/End nodes. States are labelled automatically to yield human-interpretable flows, which are visualized and evaluated using a novel automatic metric on a well-known TOD dataset, MultiWOZ 2.2. The key contributions are the unsupervised flow discovery method, the graph-based representation of dialogue dynamics, and a validation approach demonstrating substantial predictability of test transitions (over 80% at a lower threshold) without annotated data. This work enables scalable flow extraction from real dialogue histories, aiding rapid design, auditing, and explainability of TOD systems across domains.

Abstract

The design of dialogue flows is a critical but time-consuming task when developing task-oriented dialogue (TOD) systems. We propose an approach for the unsupervised discovery of flows from dialogue history, thus making the process applicable to any domain for which such an history is available. Briefly, utterances are represented in a vector space and clustered according to their semantic similarity. Clusters, which can be seen as dialogue states, are then used as the vertices of a transition graph for representing the flows visually. We present concrete examples of flows, discovered from MultiWOZ, a public TOD dataset. We further elaborate on their significance and relevance for the underlying conversations and introduce an automatic validation metric for their assessment. Experimental results demonstrate the potential of the proposed approach for extracting meaningful flows from task-oriented conversations.
Paper Structure (11 sections, 1 equation, 4 figures, 3 tables)

This paper contains 11 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Unsupervised Dialogue Flow Discovery
  4. Experimentation Setup
  5. Dataset
  6. Implementation
  7. Results
  8. Discovered Flows
  9. Automatic Evaluation
  10. Conclusion and Future Work
  11. Acknowledgements

Figures (4)

  • Figure 1: Overview of the proposed approach with three steps: utterance representation, utterance clustering, and flow discovery.
  • Figure 2: Silhouette scores for different values of $k$ for USER and SYSTEM utterances.
  • Figure 3: Flow discovered from the MultiWOZ train portion with $\theta=0.10$ and labels generated from the most frequent verb phrases.
  • Figure 4: Flow discovered from the MultiWOZ train portion with $\theta=0.15$ and labels generated from the most frequent verb phrases.