Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversations

Hao Yang, Hongyuan Lu, Xinhua Zeng, Yang Liu, Xiang Zhang, Haoran Yang, Yumeng Zhang, Shan Huang, Yiran Wei, Wai Lam

TL;DR

The paper tackles the rigidity of single-step dialogue systems by introducing Stephanie, a Step-by-Step Dialogue Paradigm that mimics ongoing human conversations. It presents a dual learning prompt framework with background information $D$, positive objectives $P$, and negative objectives $N$, optimizing $p(r|D, P, N)$, and a Further-Split post-editing method to produce more natural, multi-turn exchanges. Using PERSONA-CHAT as a base, it generates a 5,457-dialogue incremental dataset with Llama3-70b and demonstrates a plug-and-play finetuning approach to create Stephanie-enabled LLMs, outperforming single-step baselines on engagement, naturalness, and diversity. Evaluations—both automated and human—show step-by-step dialogues yield higher quality and more human-like interactions, with Stephanie achieving the strongest results. The work releases code, Stephanie datasets, and Stephanie LLMs to accelerate research and practical deployment of more natural chatbot systems.

Abstract

In the rapidly evolving field of natural language processing, dialogue systems primarily employ a single-step dialogue paradigm. Although this paradigm is efficient, it lacks the depth and fluidity of human interactions and does not appear natural. We introduce a novel \textbf{Step}-by-Step Dialogue Paradigm (Stephanie), designed to mimic the ongoing dynamic nature of human conversations. By employing a dual learning strategy and a further-split post-editing method, we generated and utilized a high-quality step-by-step dialogue dataset to fine-tune existing large language models, enabling them to perform step-by-step dialogues. We thoroughly present Stephanie. Tailored automatic and human evaluations are conducted to assess its effectiveness compared to the traditional single-step dialogue paradigm. We will release code, Stephanie datasets, and Stephanie LLMs to facilitate the future of chatbot eras.

Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversations

TL;DR

The paper tackles the rigidity of single-step dialogue systems by introducing Stephanie, a Step-by-Step Dialogue Paradigm that mimics ongoing human conversations. It presents a dual learning prompt framework with background information , positive objectives , and negative objectives , optimizing , and a Further-Split post-editing method to produce more natural, multi-turn exchanges. Using PERSONA-CHAT as a base, it generates a 5,457-dialogue incremental dataset with Llama3-70b and demonstrates a plug-and-play finetuning approach to create Stephanie-enabled LLMs, outperforming single-step baselines on engagement, naturalness, and diversity. Evaluations—both automated and human—show step-by-step dialogues yield higher quality and more human-like interactions, with Stephanie achieving the strongest results. The work releases code, Stephanie datasets, and Stephanie LLMs to accelerate research and practical deployment of more natural chatbot systems.

Abstract

In the rapidly evolving field of natural language processing, dialogue systems primarily employ a single-step dialogue paradigm. Although this paradigm is efficient, it lacks the depth and fluidity of human interactions and does not appear natural. We introduce a novel \textbf{Step}-by-Step Dialogue Paradigm (Stephanie), designed to mimic the ongoing dynamic nature of human conversations. By employing a dual learning strategy and a further-split post-editing method, we generated and utilized a high-quality step-by-step dialogue dataset to fine-tune existing large language models, enabling them to perform step-by-step dialogues. We thoroughly present Stephanie. Tailored automatic and human evaluations are conducted to assess its effectiveness compared to the traditional single-step dialogue paradigm. We will release code, Stephanie datasets, and Stephanie LLMs to facilitate the future of chatbot eras.
Paper Structure (15 sections, 1 equation, 3 figures, 6 tables)

This paper contains 15 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Dual Learning Strategy for Step-by-Step Dialogue Generation
  5. Optimizing Step-by-Step Dialogues Using the Further-Split Post-Editing Method
  6. Dataset Generation and Finetuning Strategy for Stephanie
  7. Experiment Setup
  8. Dataset
  9. Prompt
  10. Baselines and Comparison Models
  11. Evaluation Metrics
  12. Results
  13. Evaluation of Conversation Quality
  14. Fine-Tuning with Step-by-Step Dialogue
  15. Conclusion

Figures (3)

  • Figure 1: A single-step dialogue system and Stephanie. Stephanie constructs a dialogue composed of multiple dispersed yet coherent responses. ES stands for the engaging score given by humans, which is introduced in section \ref{['4.4']}.
  • Figure 2: In the process of step-by-step dialogue generation, we adopted a dual learning strategy to enhance the model's ability to generate natural dialogues through the Step-by-Step Dialogue Prompt Framework. This strategy combines positive and negative learning objectives. The positive objective includes high-quality step-by-step dialogue examples selected from real social interactions, while the negative objective comprises designed high-quality single-step dialogue examples. Through contrastive learning, this approach helps the model distinguish between step-by-step dialogues and single-step dialogues, thus generating more natural and emotionally rich step-by-step dialogues.
  • Figure 3: Distinct-N Results for Different Dialogue. This graph displays the lexical diversity of dialogues generated by various models, measured by the Distinct-N metric for n-grams from N=2 to N=6. Each colour represents a different dialogue model ($\alpha$, $\beta$, $\gamma$, Stephanie), highlighting variations in linguistic complexity and diversity.