Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systems

Layla El Asri, Hannes Schulz, Shikhar Sharma, Jeremie Zumer, Justin Harris, Emery Fine, Rahul Mehrotra, Kaheer Suleman

TL;DR

Frames introduces a large, memory-centric corpus of goal-oriented dialogues and defines frame tracking as maintaining multiple semantic frames during conversation. The paper details WOz data collection, templates, and a memory-focused annotation scheme, along with a formal definition of frames, creation/switching rules, and cross-frame references. It provides baseline NLU and frame-tracking models, showing substantial room for improvement in multi-frame reasoning and memory-aware dialogue management. The work demonstrates Frames as a platform for advancing memory-enabled dialogue systems and generation, with implications for improved decision-making and information presentation in conversational agents.

Abstract

This paper presents the Frames dataset (Frames is available at http://datasets.maluuba.com/Frames), a corpus of 1369 human-human dialogues with an average of 15 turns per dialogue. We developed this dataset to study the role of memory in goal-oriented dialogue systems. Based on Frames, we introduce a task called frame tracking, which extends state tracking to a setting where several states are tracked simultaneously. We propose a baseline model for this task. We show that Frames can also be used to study memory in dialogue management and information presentation through natural language generation.

Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systems

TL;DR

Frames introduces a large, memory-centric corpus of goal-oriented dialogues and defines frame tracking as maintaining multiple semantic frames during conversation. The paper details WOz data collection, templates, and a memory-focused annotation scheme, along with a formal definition of frames, creation/switching rules, and cross-frame references. It provides baseline NLU and frame-tracking models, showing substantial room for improvement in multi-frame reasoning and memory-aware dialogue management. The work demonstrates Frames as a platform for advancing memory-enabled dialogue systems and generation, with implications for improved decision-making and information presentation in conversational agents.

Abstract

This paper presents the Frames dataset (Frames is available at http://datasets.maluuba.com/Frames), a corpus of 1369 human-human dialogues with an average of 15 turns per dialogue. We developed this dataset to study the role of memory in goal-oriented dialogue systems. Based on Frames, we introduce a task called frame tracking, which extends state tracking to a setting where several states are tracked simultaneously. We propose a baseline model for this task. We show that Frames can also be used to study memory in dialogue management and information presentation through natural language generation.

Paper Structure

This paper contains 28 sections, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Data Collection
  4. Wizard-Of-Oz Data Collection
  5. Task Templates and Instructions
  6. Database Search Interface
  7. Suggestions
  8. Statistics of the Corpus
  9. Dialogue Annotation Scheme
  10. Dialogue Acts, Slot Types, and Slot Values
  11. Definition
  12. Creation and Annotation
  13. Research Topics
  14. Frame Tracking
  15. Definition
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Overview of the Frames corpus
  • Figure 2: Number of frames and frame switches in the corpus
  • Figure 3: Left -- number of amenities per hotel. Right -- number of hotels in the vicinity of points of interest (none if the hotel is not close to any point of interest).
  • Figure 4: Illustration of the NLU model for slots and acts prediction, taking input words and outputting labels for slots and acts. The model splits into slots-specific and acts-specific predictors after the word embedding layer, which computes a non-linearity on top of the per-word sum of character trigram embeddings.

Theorems & Definitions (1)

  • Definition 1: Frame Tracking