Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation

Shutong Feng, Hsien-chin Lin, Christian Geishauser, Nurul Lubis, Carel van Niekerk, Michael Heck, Benjamin Ruppik, Renato Vukovic, Milica Gašić

TL;DR

This work addresses the gap of incorporating emotions into task-oriented dialogue (ToD) by infusing affect into the full ToD loop: understanding, management, and generation. It builds EmoWOZ 2.0, an emotion-annotated extension that includes system conduct labels, and develops modular (EmoLoop) and end-to-end (EmoLLAMA) architectures that utilize emotion signals in dialogue state, policy, and response generation. Through extensive evaluations with simulated and human users, they demonstrate that emotion-aware ToD systems improve user sentiment and, in many settings, task success, validating the value of modelling emotions in practical dialogue systems. The work introduces an emotion-weighted RL framework, an emotion-conditioned NLG component, and an emotion-capable end-to-end model, highlighting implications for more natural, effective, and user-centric conversational agents in real-world tasks.未来方向 include richer reward structures leveraging the full emotion label set and scalable RL for LLM-based ToD systems with emotion.

Abstract

Emotions are indispensable in human communication, but are often overlooked in task-oriented dialogue (ToD) modelling, where the task success is the primary focus. While existing works have explored user emotions or similar concepts in some ToD tasks, none has so far included emotion modelling into a fully-fledged ToD system nor conducted interaction with human or simulated users. In this work, we incorporate emotion into the complete ToD processing loop, involving understanding, management, and generation. To this end, we extend the EmoWOZ dataset (Feng et al., 2022) with system affective behaviour labels. Through interactive experimentation involving both simulated and human users, we demonstrate that our proposed framework significantly enhances the user's emotional experience as well as the task success.

Infusing Emotions into Task-oriented Dialogue Systems: Understanding, Management, and Generation

TL;DR

This work addresses the gap of incorporating emotions into task-oriented dialogue (ToD) by infusing affect into the full ToD loop: understanding, management, and generation. It builds EmoWOZ 2.0, an emotion-annotated extension that includes system conduct labels, and develops modular (EmoLoop) and end-to-end (EmoLLAMA) architectures that utilize emotion signals in dialogue state, policy, and response generation. Through extensive evaluations with simulated and human users, they demonstrate that emotion-aware ToD systems improve user sentiment and, in many settings, task success, validating the value of modelling emotions in practical dialogue systems. The work introduces an emotion-weighted RL framework, an emotion-conditioned NLG component, and an emotion-capable end-to-end model, highlighting implications for more natural, effective, and user-centric conversational agents in real-world tasks.未来方向 include richer reward structures leveraging the full emotion label set and scalable RL for LLM-based ToD systems with emotion.

Abstract

Emotions are indispensable in human communication, but are often overlooked in task-oriented dialogue (ToD) modelling, where the task success is the primary focus. While existing works have explored user emotions or similar concepts in some ToD tasks, none has so far included emotion modelling into a fully-fledged ToD system nor conducted interaction with human or simulated users. In this work, we incorporate emotion into the complete ToD processing loop, involving understanding, management, and generation. To this end, we extend the EmoWOZ dataset (Feng et al., 2022) with system affective behaviour labels. Through interactive experimentation involving both simulated and human users, we demonstrate that our proposed framework significantly enhances the user's emotional experience as well as the task success.
Paper Structure (73 sections, 9 figures, 10 tables)

This paper contains 73 sections, 9 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Understanding User Emotion
  4. Dialogue Management with Emotion Feedback
  5. Generating Affective Response
  6. Simulating User Emotional Behaviour
  7. EmoWOZ 2.0: A Fully Emotion-annotated ToD Dataset
  8. Annotation Scheme
  9. Annotation Set-up
  10. Annotation Quality
  11. Infusing Emotions into ToD Systems
  12. Expanding Dialogue State with Emotion
  13. Emotion-aware Dialogue Policy
  14. Emotion Input and Output
  15. Emotion Augmented Reward
  16. ...and 58 more sections

Figures (9)

  • Figure 1: Infusing emotions into modular and end-to-end ToD systems.
  • Figure 2: RL training set-up for EmoDDPT.
  • Figure 3: The average hallucination rate of modular systems during RL training with langEmoUS. For end-to-end systems, we report hallucination rate after SL.
  • Figure 4: Average sentiment at different turn positions during language-level interaction with langEmoUS.
  • Figure 5: The average sentiment of langEmoUS during RL training of modular policy.
  • ...and 4 more figures