Controllable and Diverse Data Augmentation with Large Language Model for Low-Resource Open-Domain Dialogue Generation

TL;DR

The paper addresses data scarcity and limited semantic diversity in low-resource open-domain dialogue generation. It introduces SDA, a three-stage framework that uses seed dialogue summaries as planning prompts to guide LLM-generated augmentation, producing high-quality and semantically diverse dialogues while aligning with the seed data distribution. A novel SemanticDiversity metric is proposed to quantify semantic-level diversity, and experiments on DailyDialog show that SDA improves both data quality and downstream model performance, with ablations confirming the importance of filtering and bootstrapped summary augmentation. Overall, SDA demonstrates controllable, distribution-aware data augmentation for low-resource dialogue settings and provides a reproducible pipeline and diversity metric to guide future research.

Abstract

Data augmentation (DA) is crucial to mitigate model training instability and over-fitting problems in low-resource open-domain dialogue generation. However, traditional DA methods often neglect semantic data diversity, restricting the overall quality. Recently, large language models (LLM) have been used for DA to generate diversified dialogues. However, they have limited controllability and tend to generate dialogues with a distribution shift compared to the seed dialogues. To maximize the augmentation diversity and address the controllability problem, we propose \textbf{S}ummary-based \textbf{D}ialogue \textbf{A}ugmentation with LLM (SDA). Our approach enhances the controllability of LLM by using dialogue summaries as a planning tool. Based on summaries, SDA can generate high-quality and diverse dialogue data even with a small seed dataset. To evaluate the efficacy of data augmentation methods for open-domain dialogue, we designed a clustering-based metric to characterize the semantic diversity of the augmented dialogue data. The experimental results show that SDA can augment high-quality and semantically diverse dialogues given a small seed dataset and an LLM, and the augmented data can boost the performance of open-domain dialogue models.

Figures (5)

• Figure 1: Overview of traditional data augmentation method and Summary-based Dialogue Augmentation with Llm (SDA).
• Figure 2: Main framework of our method. The process is in three steps: (1) First, summarizing the seed dialogue into dialogue summary. (2) Secondly, we leverage the seed dialogue summary to generate more dialogue summaries with a wide diversity of topics. (3) Finally, we convert the augmented dialogue summary back into dialogue. All these steps are performed by Llm.
• Figure 3: Dialogue perplexity distribution with different data augmentation methods (best viewed in color).
• Figure 4: The t-SNE visualization of augmented dialogues.
• Figure 5: The performance of various data augmentation methods given 100/200/500 seed dialogues (best viewed in color). The initial two (PPL, SD) are metrics for evaluating the augmented data, while the latter four are metrics for evaluating the predictions of the model trained on the augmented data.