PARL-MT: Learning to Call Functions in Multi-Turn Conversation with Progress Awareness

TL;DR

This work tackles the challenge of multi-turn function calling by injecting explicit progress awareness into both data synthesis and training. It introduces PARL-MT, a two-phase framework consisting of Progress Awareness Generation (PAG) to produce history-aware, future-planning data, and Progress Awareness-Guided RL (PAG-RL) to train with awareness-guided rollouts and a composite reward. Across two public benchmarks and multiple backbone LLMs, PARL-MT consistently surpasses baselines, with ablation studies confirming the substantial contribution of the progress-awareness components. The approach reduces contextual redundancy, aligns local actions with long-horizon goals, and demonstrates practical benefits in real multi-turn dialogues, though it incurs additional computation during rollout that future work can address through more efficient awareness components and alternative RL algorithms.

Abstract

Large language models (LLMs) have achieved impressive success in single-turn function calling, yet real-world applications such as travel planning or multi-stage data analysis typically unfold across multi-turn conversations. In these settings, LLMs must not only issue accurate function calls at each step but also maintain progress awareness, the ability to summarize past interactions and plan future actions to ensure coherent, long-horizon task execution. Existing approaches, however, either reduce multi-turn training to isolated single-turn samples, which neglects task-level planning, or employ end-to-end reinforcement learning (RL) that struggles with redundancy and lacks explicit integration of progress awareness. To overcome these limitations, we introduce PARL-MT, a framework that explicitly incorporates progress awareness into LLM training for multi-turn function calling. PARL-MT combines (i) a Progress Awareness Generation (PAG) pipeline, which automatically constructs datasets coupling conversation summaries with future task planning, and (ii) a Progress Awareness-Guided Reinforcement Learning (PAG-RL) algorithm, which integrates progress awareness into RL training to reduce contextual redundancy and improve alignment between local actions and global task completion. Empirical results on two public benchmarks demonstrate that PARL-MT significantly outperforms existing methods, highlighting the effectiveness of progress awareness in enabling robust and efficient multi-turn function calling.

Figures (10)

• Figure 1: Limitations of single-turn training reside in overlooking progress in conversations
• Figure 2: Overview of PAG. By summarizing history and planning future in Step 2, PAG generates progress awareness for each conversation turn and constructs a high-quality awareness dataset.
• Figure 3: Overview of PAG-RL. By incorporating progress awareness, the policy model generates multi-step action trajectories in response to a query.
• Figure 3: Quantitative comparison.
• Figure 4: Awareness capability across phases