Zero-Shot Generalizable End-to-End Task-Oriented Dialog System using Context Summarization and Domain Schema

TL;DR

This work tackles the bottleneck of labeled data when scaling task-oriented dialog (ToD) systems to new domains by introducing ZS-ToD, a zero-shot end-to-end ToD framework that leverages domain schemas and concise dialog-state summarization. The model builds on GPT-2 and uses a two-step training regimen: first learn the general structure of dialog data, then optimize the actual response generation and intermediate outputs, all conditioned on domain schemas. By replacing long dialog histories with a compact DS representation and conditioning on domain schemas, ZS-ToD achieves strong zero-shot generalization across 20 SGD domains and the SGD-X variants, outperforming baselines on key metrics such as joint goal accuracy and inform. The approach reduces data collection burdens for new domains and demonstrates robustness to schema variations, with ablations confirming the critical roles of schema guidance and the two-step training strategy.

Abstract

Task-oriented dialog systems empower users to accomplish their goals by facilitating intuitive and expressive natural language interactions. State-of-the-art approaches in task-oriented dialog systems formulate the problem as a conditional sequence generation task and fine-tune pre-trained causal language models in the supervised setting. This requires labeled training data for each new domain or task, and acquiring such data is prohibitively laborious and expensive, thus making it a bottleneck for scaling systems to a wide range of domains. To overcome this challenge, we introduce a novel Zero-Shot generalizable end-to-end Task-oriented Dialog system, ZS-ToD, that leverages domain schemas to allow for robust generalization to unseen domains and exploits effective summarization of the dialog history. We employ GPT-2 as a backbone model and introduce a two-step training process where the goal of the first step is to learn the general structure of the dialog data and the second step optimizes the response generation as well as intermediate outputs, such as dialog state and system actions. As opposed to state-of-the-art systems that are trained to fulfill certain intents in the given domains and memorize task-specific conversational patterns, ZS-ToD learns generic task-completion skills by comprehending domain semantics via domain schemas and generalizing to unseen domains seamlessly. We conduct an extensive experimental evaluation on SGD and SGD-X datasets that span up to 20 unique domains and ZS-ToD outperforms state-of-the-art systems on key metrics, with an improvement of +17% on joint goal accuracy and +5 on inform. Additionally, we present a detailed ablation study to demonstrate the effectiveness of the proposed components and training mechanism

Figures (5)

• Figure 1: Overview of $\mathsf{ZS}\hbox{-}\mathsf{ToD}$: the domain schema facilitates estimating dialog state, system actions, and system response irrespective of whether the model was trained on that domain or not. Parts of the schema that assist in the generation are grouped by similar colors.
• Figure 2: Overview of our approach. A GPT-2 model is fed the dialog state of the previous turn, the last user utterance, relevant schemas, database search results, and a list of system action names. As output, the model autoregressively generates the current dialog state, user actions, system actions, and system response.
• Figure 3: Performance of dialog systems on the SGD test set with respect to dialog turns
• Figure 4: Effect of Two Step Training on dialog systems
• Figure 5: SGD-X results: Mean and standard deviation of each metric across all versions of SGD-X