Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems
Andrea Madotto, Zihan Liu, Zhaojiang Lin, Pascale Fung
TL;DR
The paper tackles data efficiency in task-oriented dialogue by assessing few-shot learning via prompting large language models for four core tasks: NLU, DST, ACT, and NLG. It introduces LM priming with fixed prefixes (binary, value-based, generative) that avoids parameter updates, enabling a single LM to handle multiple tasks. Experimental results show that larger GPT-2 variants improve NLU and NLG performance and can reach or exceed weak finetuning baselines in some settings, while DST and ACT show more variable gains and remain challenging. The authors identify major limitations—context window size and computational cost due to multiple forwards—and propose future work on longer-context models and end-to-end dialogue benchmarks to extend this approach.
Abstract
Task-oriented dialogue systems use four connected modules, namely, Natural Language Understanding (NLU), a Dialogue State Tracking (DST), Dialogue Policy (DP) and Natural Language Generation (NLG). A research challenge is to learn each module with the least amount of samples (i.e., few-shots) given the high cost related to the data collection. The most common and effective technique to solve this problem is transfer learning, where large language models, either pre-trained on text or task-specific data, are fine-tuned on the few samples. These methods require fine-tuning steps and a set of parameters for each task. Differently, language models, such as GPT-2 (Radford et al., 2019) and GPT-3 (Brown et al., 2020), allow few-shot learning by priming the model with few examples. In this paper, we evaluate the priming few-shot ability of language models in the NLU, DST, DP and NLG tasks. Importantly, we highlight the current limitations of this approach, and we discuss the possible implication for future work.