Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System

Chen Chen, Ruizhe Li, Yuchen Hu, Yuanyuan Chen, Chengwei Qin, Qiang Zhang

TL;DR

This work tackles catastrophic forgetting in continual learning for task-oriented dialogue systems by introducing HESIT, a Hessian-free method that selects influential exemplars via hyper-gradient tracing to inform rehearsal. By estimating data influence on unseen validation data without computing inverse Hessians, HESIT maintains a compact replay buffer and achieves state-of-the-art performance on the largest ToDs CL benchmark (37 domains). The key contributions are the hyper-gradient-based exemplar selection, a Hessian-free training schedule, and extensive empirical validation showing improved INTENT accuracy, JGA, EER, and BLEU across both E2E and modular ToDs. This approach enables scalable continual learning in ToDs with large pre-trained models, offering practical benefits for real-time, multi-domain dialogue systems while acknowledging ethical considerations and potential cross-domain extensions.

Abstract

Intelligent task-oriented dialogue systems (ToDs) are expected to continuously acquire new knowledge, also known as Continual Learning (CL), which is crucial to fit ever-changing user needs. However, catastrophic forgetting dramatically degrades the model performance in face of a long streamed curriculum. In this paper, we aim to overcome the forgetting problem in ToDs and propose a method (HESIT) with hyper-gradient-based exemplar strategy, which samples influential exemplars for periodic retraining. Instead of unilaterally observing data or models, HESIT adopts a profound exemplar selection strategy that considers the general performance of the trained model when selecting exemplars for each task domain. Specifically, HESIT analyzes the training data influence by tracing their hyper-gradient in the optimization process. Furthermore, HESIT avoids estimating Hessian to make it compatible for ToDs with a large pre-trained model. Experimental results show that HESIT effectively alleviates catastrophic forgetting by exemplar selection, and achieves state-of-the-art performance on the largest CL benchmark of ToDs in terms of all metrics.

Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System

TL;DR

This work tackles catastrophic forgetting in continual learning for task-oriented dialogue systems by introducing HESIT, a Hessian-free method that selects influential exemplars via hyper-gradient tracing to inform rehearsal. By estimating data influence on unseen validation data without computing inverse Hessians, HESIT maintains a compact replay buffer and achieves state-of-the-art performance on the largest ToDs CL benchmark (37 domains). The key contributions are the hyper-gradient-based exemplar selection, a Hessian-free training schedule, and extensive empirical validation showing improved INTENT accuracy, JGA, EER, and BLEU across both E2E and modular ToDs. This approach enables scalable continual learning in ToDs with large pre-trained models, offering practical benefits for real-time, multi-domain dialogue systems while acknowledging ethical considerations and potential cross-domain extensions.

Abstract

Intelligent task-oriented dialogue systems (ToDs) are expected to continuously acquire new knowledge, also known as Continual Learning (CL), which is crucial to fit ever-changing user needs. However, catastrophic forgetting dramatically degrades the model performance in face of a long streamed curriculum. In this paper, we aim to overcome the forgetting problem in ToDs and propose a method (HESIT) with hyper-gradient-based exemplar strategy, which samples influential exemplars for periodic retraining. Instead of unilaterally observing data or models, HESIT adopts a profound exemplar selection strategy that considers the general performance of the trained model when selecting exemplars for each task domain. Specifically, HESIT analyzes the training data influence by tracing their hyper-gradient in the optimization process. Furthermore, HESIT avoids estimating Hessian to make it compatible for ToDs with a large pre-trained model. Experimental results show that HESIT effectively alleviates catastrophic forgetting by exemplar selection, and achieves state-of-the-art performance on the largest CL benchmark of ToDs in terms of all metrics.
Paper Structure (24 sections, 9 equations, 4 figures, 7 tables, 2 algorithms)

This paper contains 24 sections, 9 equations, 4 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Background: End-to-end ToDs
  5. Rehearsal-based CL in ToDs
  6. HESIT: Exemplar Selection
  7. HESIT: Training Schedule
  8. Experiment
  9. Dataset
  10. Evaluation Metrics
  11. Baselines
  12. Experimental Setup
  13. Result and Analysis
  14. Main Comparison Results
  15. Analysis for Exemplar Selection
  16. ...and 9 more sections

Figures (4)

  • Figure 1: (a) Rehearsal-based CL in Task-oriented dialogue system. Exemplars $E_t$ are sampled from $t$-domain training data for episodic rehearsal. (b) Exemplar selection in terms of influence chain "Data (I) -- Model (II) -- Performance (III)". Our method penetrates into the performance perspective.
  • Figure 2: Learning curve for INTENT accuracy in E2E setting. Each test point is evaluated on the already learned task in the curriculum.
  • Figure 3: Inter-class and Intra-class (diagonal elements) contributions for CIFAR-10 dataset, which are measured by (a) LISSA and (b) HESIT.
  • Figure 4: Example of input-out pairs, for the four settings, INTENT, DST, NLG and end-to-end (E2E).