LLM-Driven Preference Data Synthesis for Proactive Prediction of the Next User Utterance in Human-Machine Dialogue

TL;DR

ProUtt tackles privacy- and compute-aware proactive next-utterance prediction by synthesizing data with explicit user-intent reasoning encoded as hierarchical intent trees. It combines sentence-type reasoning, exploitation-exploration path reasoning, and revision/perturbation to generate preference and non-preference trajectories, which effectively train compact LLMs for proactive dialogue. Empirical results across four datasets show ProUtt outperforms commercial APIs, user simulators, and prior synthesis methods on both pointwise and pairwise evaluations, with robust performance across backbones and tasks. The work provides openly available code and datasets, highlighting practical benefits for privacy-preserving, data-efficient training of task-specific dialogue models.

Abstract

Proactively predicting a users next utterance in human-machine dialogue can streamline interaction and improve user experience. Existing commercial API-based solutions are subject to privacy concerns while deploying general-purpose LLMs locally remains computationally expensive. As such, training a compact, task-specific LLM provides a practical alternative. Although user simulator methods can predict a user's next utterance, they mainly imitate their speaking style rather than advancing the dialogue. Preference data synthesis has been investigated to generate data for proactive next utterance prediction and help align LLMs with user preferences. Yet existing methods lack the ability to explicitly model the intent reasoning that leads to the user's next utterance and to define and synthesize preference and non-preference reasoning processes for predicting the user's next utterance.To address these challenges, we propose ProUtt, an LLM-driven preference data synthesis method for proactive next utterance prediction. ProUtt converts dialogue history into an intent tree and explicitly models intent reasoning trajectories by predicting the next plausible path from both exploitation and exploration perspectives. It then constructs preference and non-preference reasoning processes by perturbing or revising intent tree paths at different future turns. Extensive evaluations using LLM-as-a-judge and human judgments demonstrate that ProUtt consistently outperforms existing data synthesis methods, user simulators, and commercial LLM APIs across four benchmark datasets. We release both the code and the synthesized datasets to facilitate future research.

Figures (9)

• Figure 1: The intent tree extraction process. The LLM converts a human–LLM dialogue into a user intent tree and predicts the next path on the tree from both exploitation and exploration perspectives.
• Figure 2: Overview of the ProUtt pipeline, showing hierarchical intent tree construction, sentence type reasoning, intent path reasoning, and the generation of preferred and non-preferred reasoning processes. “Similar Positive Intent” denotes cases where the generated user utterance aligns with the ground-truth intent at a semantic level, without requiring exact surface-form matching.
• Figure 3: Pairwise win–tie–loss comparison of ProUtt and baselines on four dialogue datasets under LLM and human evaluation.
• Figure 4: Human--LLM evaluation consistency across four datasets. The bar plots report Agreement Rate and Cohen's $\kappa$ between LLM and human judgments on LMSYS, ShareGPT, WildChat, and CrossWOZ. Error bars indicate 95% confidence intervals for both Agreement Rate and Cohen's $\kappa$.
• Figure 5: Confusion matrices comparing human and LLM pairwise judgments across four datasets, illustrating their agreement over win, tie, and loss outcomes. Darker diagonal cells indicate higher agreement and thus stronger consistency between LLM-based and human evaluations.