Controllable Discovery of Intents: Incremental Deep Clustering Using Semi-Supervised Contrastive Learning

TL;DR

The Controllable Discovery of Intents (CDI) framework domain and prior knowledge are incorporated using a sequence of unsupervised contrastive learning on unlabeled data followed by fine-tuning on partially labeled data, and finally iterative refinement of clustering and representations through repeated clustering and pseudo-label fine-tuning.

Abstract

Deriving value from a conversational AI system depends on the capacity of a user to translate the prior knowledge into a configuration. In most cases, discovering the set of relevant turn-level speaker intents is often one of the key steps. Purely unsupervised algorithms provide a natural way to tackle discovery problems but make it difficult to incorporate constraints and only offer very limited control over the outcomes. Previous work has shown that semi-supervised (deep) clustering techniques can allow the system to incorporate prior knowledge and constraints in the intent discovery process. However they did not address how to allow for control through human feedback. In our Controllable Discovery of Intents (CDI) framework domain and prior knowledge are incorporated using a sequence of unsupervised contrastive learning on unlabeled data followed by fine-tuning on partially labeled data, and finally iterative refinement of clustering and representations through repeated clustering and pseudo-label fine-tuning. In addition, we draw from continual learning literature and use learning-without-forgetting to prevent catastrophic forgetting across those training stages. Finally, we show how this deep-clustering process can become part of an incremental discovery strategy with human-in-the-loop. We report results on both CLINC and BANKING datasets. CDI outperforms previous works by a significant margin: 10.26% and 11.72% respectively.

Figures (5)

• Figure 1: A sample dialogue between an agent and the customer from the banking domain along with the demonstration of the intent discovery process (CDI).
• Figure 2: Our proposed architecture. We begin by training the MPNet model with unsupervised contrastive loss (UCL) on the unlabeled dataset, followed by a two-stage training process along with LwF.
• Figure 3: Our proposed architecture for incremental intent discovery via human-in-the-loop involves utilizing a pre-trained model on a labeled dataset for generating representations using unsupervised contrastive learning (UCL). Then we perform K-means, and the user is presented with the clusters for input. The user can provide labeled samples and newly discovered intents by selecting or deselecting samples. Stage-1 and stage-2 training are performed using the labeled and unlabeled datasets along with LwF, and the process is iterated until no new intents are discovered.
• Figure 4: Effectiveness of Known Ratio on three datasets.
• Figure 5: TSNE plots at different iterations for TELECOM dataset.