User-Oriented Multi-Turn Dialogue Generation with Tool Use at scale

TL;DR

The paper tackles the challenge of training large reasoning models for open-ended, multi-turn tool use in realistic human–agent collaboration, identifying the limitations of static toolsets and single-shot data. It introduces a user-oriented simulation paradigm that decouples tasks from interaction, employing a plug-and-play data generator and an execution-grounded SQL tool environment to produce high-density, authentic dialogues. The proposed three-component pipeline—dynamic tool and task synthesis, plug-and-play scalability, and high-density trajectories—yields substantial improvements on long-horizon benchmarks (BFCL and tau), with enhanced consistency in tool usage. While the approach increases generation costs and tightens environment-dependency, the results suggest execution-grounded supervision as a robust path toward faithful tool selection, persistent state tracking, and realistic user-agent interactions across diverse domains.

Abstract

The recent paradigm shift toward large reasoning models (LRMs) as autonomous agents has intensified the demand for sophisticated, multi-turn tool-use capabilities. Yet, existing datasets and data-generation approaches are limited by static, predefined toolsets that cannot scale to the complexity of open-ended human-agent collaboration. To address this, we initially developed a framework for automated task-oriented multi-turn dialogue generation at scale, utilizing an LRM-based simulator to dynamically generate high-value, domain-specific tools to solve specified tasks. However, we observe that a purely task-oriented design often results in "solely task-solving" trajectories, where the agent completes the objective with minimal interaction, failing to generate the high turn-count conversations seen in realistic scenarios. To bridge this gap, we shift toward a user-oriented simulation paradigm. By decoupling task generation from a dedicated user simulator that mimics human behavioral rules - such as incremental request-making and turn-by-turn feedback - we facilitate more authentic, extended multi-turn dialogues that reflect the iterative nature of real-world problem solving. Our generation pipeline operates as a versatile, plug-and-play module capable of initiating generation from any state, ensuring high scalability in producing extended tool-use data. Furthermore, by facilitating multiple task completions within a single trajectory, it yields a high-density dataset that reflects the multifaceted demands of real-world human-agent interaction.

Figures (8)

• Figure 1: Plug-and-Play Tool Preparation Module. A modular pipeline for dynamic tool synthesis and preprocessing, designed to initiate multi-turn data generation from any arbitrary state.
• Figure 2: Task-Oriented Multi-Turn Conversation Generation Pipeline. An automated framework that generates tool-use trajectories focused on efficient task completion through direct simulator-based responses.
• Figure 3: User-Oriented Multi-Turn Conversation Generation Pipeline. A framework that decouples tasks from interaction by employing a dedicated user simulator to mimic incremental human feedback and request-making.
• Figure 4: User-oriented Tool-Execution Multi-turn Conversation Generation Pipeline. This pipeline integrates a SQL-tool generation module grounded in real-world database schemas with a dedicated user simulator to produce verifiable, high-fidelity multi-turn dialogues.
• Figure 5: Consistency analysis across varying $k$ values. The charts illustrate the Pass$^$k performance for different models (GPT-OSS-120b, Qwen3-4b, and Qwen3-30b) across the Retail, Airline, and Telecom domains, showing how performance scales preserve in overall domains while increased $k$ values.