Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Dimensional Prompt Chaining to Improve Open-Domain Dialogue Generation

Livia Leong Hui Teng

TL;DR

This work tackles the quality gap between small language models and large models in open-domain dialogue. It introduces a multidimensional prompt-chaining framework that sequentially refines responses along coherence, engagingness, and naturalness using in-context learning prompts. Across TinyLlama and Llama-2-7B, the method yields substantial improvements in diversity, coherence, and naturalness, with results approaching those of much larger models such as Llama-2-70B and GPT-3.5 Turbo. Ablation studies reveal model-dependent interactions between dimensions, yet the full framework consistently delivers the highest-quality, human-like dialogue while remaining computationally efficient.

Abstract

Small language models (SLMs) offer significant deployment advantages but often struggle to match the dialogue quality of larger models in open-domain settings. In this paper, we propose a multi-dimensional prompt-chaining framework that integrates Naturalness, Coherence, and Engagingness dimensions to enhance human-likeness in open-domain dialogue generation. We apply the framework to two SLMs, TinyLlama and Llama-2-7B, and benchmark their performance against responses generated by substantially larger models, including Llama-2-70B and GPT-3.5 Turbo. We then employ automatic and human evaluation to assess the responses based on diversity, contextual coherence, as well as overall quality. Results show that the full framework improves response diversity by up to 29%, contextual coherence by up to 28%, and engagingness as well as naturalness by up to 29%. Notably, Llama-2-7B achieves performance comparable to substantially larger models, including Llama-2-70B and GPT-3.5 Turbo. Overall, the findings demonstrate that carefully designed prompt-based strategies provide an effective and resource-efficient pathway to improving open-domain dialogue quality in SLMs.

Multi-Dimensional Prompt Chaining to Improve Open-Domain Dialogue Generation

TL;DR

This work tackles the quality gap between small language models and large models in open-domain dialogue. It introduces a multidimensional prompt-chaining framework that sequentially refines responses along coherence, engagingness, and naturalness using in-context learning prompts. Across TinyLlama and Llama-2-7B, the method yields substantial improvements in diversity, coherence, and naturalness, with results approaching those of much larger models such as Llama-2-70B and GPT-3.5 Turbo. Ablation studies reveal model-dependent interactions between dimensions, yet the full framework consistently delivers the highest-quality, human-like dialogue while remaining computationally efficient.

Abstract

Small language models (SLMs) offer significant deployment advantages but often struggle to match the dialogue quality of larger models in open-domain settings. In this paper, we propose a multi-dimensional prompt-chaining framework that integrates Naturalness, Coherence, and Engagingness dimensions to enhance human-likeness in open-domain dialogue generation. We apply the framework to two SLMs, TinyLlama and Llama-2-7B, and benchmark their performance against responses generated by substantially larger models, including Llama-2-70B and GPT-3.5 Turbo. We then employ automatic and human evaluation to assess the responses based on diversity, contextual coherence, as well as overall quality. Results show that the full framework improves response diversity by up to 29%, contextual coherence by up to 28%, and engagingness as well as naturalness by up to 29%. Notably, Llama-2-7B achieves performance comparable to substantially larger models, including Llama-2-70B and GPT-3.5 Turbo. Overall, the findings demonstrate that carefully designed prompt-based strategies provide an effective and resource-efficient pathway to improving open-domain dialogue quality in SLMs.
Paper Structure (12 sections, 2 equations, 2 figures, 2 tables)

This paper contains 12 sections, 2 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Initial Response Generation
  4. Stage 1: Coherence Evaluation
  5. Stage 2: Enhancing Engagingness
  6. Stage 3: Enhancing Naturalness
  7. Experimental Design
  8. Ablation Study
  9. Evaluation Metrics
  10. Results and Discussion
  11. Related Work
  12. Conclusion

Figures (2)

  • Figure 1: Workflow of the response generation framework. The process includes: initial response generation, (1) coherence evaluation with up to $k$ iterations, (2) engagingness improvement if coherence is achieved and (3) naturalness improvement to finalize the response.
  • Figure 2: Prompt templates for Stage 1,2 and 3 of the pipeline.