Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Is one brick enough to break the wall of spoken dialogue state tracking?

Lucas Druart, Valentin Vielzeuf, Yannick Estève

TL;DR

This work investigates spoken Task-Oriented Dialogue DST by comparing a traditional cascade pipeline against a completely neural end-to-end approach, under equal resource constraints. It formalizes DST in TOD as updating $DS_t$ from the previous state $DS_{t-2}$ and recent turns, and evaluates how context propagation strategies affect performance. The completely neural model uses joint audio-text embeddings with a fusion mechanism to predict $DS_t$, showing competitive results to cascade in audio-native settings like SpokenWOZ, while cascade generally remains stronger on standard benchmarks. The findings underscore the potential of end-to-end spoken DST but also highlight ongoing challenges in propagating and utilizing past context; future work includes training techniques that explicitly model the uncertainty in prior context. Overall, the paper provides a path toward fully neural spoken DST and emphasizes context-aware training as a promising direction for robust dialogue state updates.

Abstract

In Task-Oriented Dialogue (TOD) systems, correctly updating the system's understanding of the user's requests (\textit{a.k.a} dialogue state tracking) is key to a smooth interaction. Traditionally, TOD systems perform this update in three steps: transcription of the user's utterance, semantic extraction of the key concepts, and contextualization with the previously identified concepts. Such cascade approaches suffer from cascading errors and separate optimization. End-to-End approaches have been proven helpful up to the turn-level semantic extraction step. This paper goes one step further and provides (1) a novel approach for completely neural spoken DST, (2) an in depth comparison with a state of the art cascade approach and (3) avenues towards better context propagation. Our study highlights that jointly-optimized approaches are also competitive for contextually dependent tasks, such as Dialogue State Tracking (DST), especially in audio native settings. Context propagation in DST systems could benefit from training procedures accounting for the previous' context inherent uncertainty.

Is one brick enough to break the wall of spoken dialogue state tracking?

TL;DR

This work investigates spoken Task-Oriented Dialogue DST by comparing a traditional cascade pipeline against a completely neural end-to-end approach, under equal resource constraints. It formalizes DST in TOD as updating from the previous state and recent turns, and evaluates how context propagation strategies affect performance. The completely neural model uses joint audio-text embeddings with a fusion mechanism to predict , showing competitive results to cascade in audio-native settings like SpokenWOZ, while cascade generally remains stronger on standard benchmarks. The findings underscore the potential of end-to-end spoken DST but also highlight ongoing challenges in propagating and utilizing past context; future work includes training techniques that explicitly model the uncertainty in prior context. Overall, the paper provides a path toward fully neural spoken DST and emphasizes context-aware training as a promising direction for robust dialogue state updates.

Abstract

In Task-Oriented Dialogue (TOD) systems, correctly updating the system's understanding of the user's requests (\textit{a.k.a} dialogue state tracking) is key to a smooth interaction. Traditionally, TOD systems perform this update in three steps: transcription of the user's utterance, semantic extraction of the key concepts, and contextualization with the previously identified concepts. Such cascade approaches suffer from cascading errors and separate optimization. End-to-End approaches have been proven helpful up to the turn-level semantic extraction step. This paper goes one step further and provides (1) a novel approach for completely neural spoken DST, (2) an in depth comparison with a state of the art cascade approach and (3) avenues towards better context propagation. Our study highlights that jointly-optimized approaches are also competitive for contextually dependent tasks, such as Dialogue State Tracking (DST), especially in audio native settings. Context propagation in DST systems could benefit from training procedures accounting for the previous' context inherent uncertainty.
Paper Structure (13 sections, 1 equation, 4 figures, 1 table)

This paper contains 13 sections, 1 equation, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Task-Oriented Dialogues
  4. Context Propagation
  5. Traditional Cascade Approach
  6. Completely Neural Approach
  7. Results
  8. Datasets
  9. Evaluation
  10. Slot group analysis
  11. Dialogue turn analysis
  12. Discussion
  13. Conclusion

Figures (4)

  • Figure 1: Spoken Dialogue State Tracking alternatives. Red characters indicate potential cascading errors.
  • Figure 2: Two approaches for context propagation in spoken DST: SOTA cascade (top) and completely neural models (bottom). The inputs are displayed in the middle: agent previous turn $A_{t-1}$, user current turn $U_{t}$ and previous dialogue state $DS_{t-2}$. The output is the current dialogue state $DS_t$. Hatched components are speech-related while solid ones are text-related. Colored blocks are fine-tuned while white ones are trained from scratch.
  • Figure 3: Slot group average F1
  • Figure 4: Turn accuracy with and without ground-truth previous state for each approach. Note that there are fewer and fewer dialogues as the number of turns increases.