Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications

Sathya Krishnan Suresh, Wu Mengjun, Tushar Pranav, Eng Siong Chng

TL;DR

DiaSynth addresses domain-specific dialogue data scarcity by generating synthetic dialogues with LLMs guided by Chain of Thought, subtopics, and persona conditioning. The framework achieves contextually rich conversations and scalable data generation, releasing code and data openly. Empirical results show that fine-tuning on DiaSynth data improves downstream dialogue summarization and can approach in-domain performance, with about 90.5% coverage relative to real-domain data. Open-source LLMs like LLaMA-3 excel on informal dialogues, while GPT-4o performs well on more formal structures, illustrating complementary strengths of model families. The work highlights a practical, scalable route to enriching low-resource dialogue domains, while also noting hallucination and topic-coverage limitations that warrant further study.

Abstract

The scarcity of domain-specific dialogue datasets limits the development of dialogue systems across applications. Existing research is constrained by general or niche datasets that lack sufficient scale for training dialogue systems. To address this gap, we introduce DiaSynth - a synthetic dialogue generation framework capable of generating high-quality, contextually rich dialogues across a wide range of domains. Unlike existing frameworks, DiaSynth uses Large Language Models (LLMs) and Chain of Thought (CoT) reasoning to generate dynamic, domain-specific dialogues with simulated personas and diverse conversational features. We perform our experiments by generating synthetic data using different LLMs and few-shot examples from DialogSum and SAMSum. The pretrained language models fine-tuned on the synthetic data outperform the base models by 16.47% on dialogue summarization, while the comparison between models fine-tuned on in-domain data and synthetic data shows that the synthetic data is able to capture 90.48% of the performance distribution of the in-domain data on dialogue summarization. The quality of the data generated also increases as we increase the size of LLM from 3B to 8B. These results validate DiaSynth's potential as a robust alternative to traditional data collection methods. We open source the code and data generated for future research.

DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications

TL;DR

DiaSynth addresses domain-specific dialogue data scarcity by generating synthetic dialogues with LLMs guided by Chain of Thought, subtopics, and persona conditioning. The framework achieves contextually rich conversations and scalable data generation, releasing code and data openly. Empirical results show that fine-tuning on DiaSynth data improves downstream dialogue summarization and can approach in-domain performance, with about 90.5% coverage relative to real-domain data. Open-source LLMs like LLaMA-3 excel on informal dialogues, while GPT-4o performs well on more formal structures, illustrating complementary strengths of model families. The work highlights a practical, scalable route to enriching low-resource dialogue domains, while also noting hallucination and topic-coverage limitations that warrant further study.

Abstract

The scarcity of domain-specific dialogue datasets limits the development of dialogue systems across applications. Existing research is constrained by general or niche datasets that lack sufficient scale for training dialogue systems. To address this gap, we introduce DiaSynth - a synthetic dialogue generation framework capable of generating high-quality, contextually rich dialogues across a wide range of domains. Unlike existing frameworks, DiaSynth uses Large Language Models (LLMs) and Chain of Thought (CoT) reasoning to generate dynamic, domain-specific dialogues with simulated personas and diverse conversational features. We perform our experiments by generating synthetic data using different LLMs and few-shot examples from DialogSum and SAMSum. The pretrained language models fine-tuned on the synthetic data outperform the base models by 16.47% on dialogue summarization, while the comparison between models fine-tuned on in-domain data and synthetic data shows that the synthetic data is able to capture 90.48% of the performance distribution of the in-domain data on dialogue summarization. The quality of the data generated also increases as we increase the size of LLM from 3B to 8B. These results validate DiaSynth's potential as a robust alternative to traditional data collection methods. We open source the code and data generated for future research.
Paper Structure (40 sections, 4 equations, 3 figures, 20 tables, 1 algorithm)

This paper contains 40 sections, 4 equations, 3 figures, 20 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Personality in Synthetic Data Generation
  4. Prompting Task-Oriented Dialogue Generation
  5. Existing Task-Oriented Dialogue Datasets
  6. DiaSynth
  7. Subtopic Generation
  8. Persona Generation
  9. Dialogue Generation
  10. Experimental Setup
  11. Quality of the dialogues
  12. Downstream Task - Summarization
  13. Results
  14. Quality of the Dialogues
  15. Metric Scores
  16. ...and 25 more sections

Figures (3)

  • Figure 1: Overview of DiaSynth
  • Figure 2: CoT Example 1
  • Figure 3: CoT Example 2