Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model

TL;DR

This work identifies a major gap between task-oriented dialogue systems and language agents, and proposes CoALM, a unified Conversational Agentic Language Model, trained via a hybrid CoALM-IT dataset that interleaves TOD, LA, and CRA ( Conversational ReAct API) data. By fine-tuning open-source Llama-based models (8B, 70B, 405B) with LoRA in a zero-shot setting, CoALM achieves state-of-the-art performance across TOD and function-calling benchmarks, surpassing GPT-4o on several metrics. The results demonstrate the feasibility of a single model mastering both multi-turn dialogue management and complex tool use, supported by ablations showing the importance of each data component. The work releases data, code, and models to encourage open research and practical development of versatile, open-source conversational agents.

Abstract

Large Language Models (LLMs) with API-calling capabilities enabled building effective Language Agents (LA), while also revolutionizing the conventional task-oriented dialogue (TOD) paradigm. However, current approaches face a critical dilemma: TOD systems are often trained on a limited set of target APIs, requiring new data to maintain their quality when interfacing with new services, while LAs are not trained to maintain user intent over multi-turn conversations. Because both robust multi-turn management and advanced function calling are crucial for effective conversational agents, we evaluate these skills on three popular benchmarks: MultiWOZ 2.4 (TOD), BFCL V3 (LA), and API-Bank (LA), and our analyses reveal that specialized approaches excel in one domain but underperform in the other. To bridge this chasm, we introduce CoALM (Conversational Agentic Language Model), a unified approach that integrates both conversational and agentic capabilities. We created CoALM-IT, a carefully constructed multi-task dataset that interleave multi-turn ReAct reasoning with complex API usage. Using CoALM-IT, we train three models CoALM 8B, CoALM 70B, and CoALM 405B, which outperform top domain-specific models, including GPT-4o, across all three benchmarks. This demonstrates the feasibility of a single model approach for both TOD and LA, setting a new standard for conversational agents.

Figures (7)

• Figure 1: Unifying Capabilities of TOD Systems and LAs. TOD systems excel in multi-turn conversations and task completion but lack advanced API capabilities, while LA handle APIs well but struggle with coherent multi-turn dialogue.
• Figure 2: Overview of the CoALM Pipeline. This figure illustrates our dataset generation and fine-tuning framework. The top three rows depict the data transformation processes, along with a corresponding sample shown on the right. In each training sample, green text highlights the input components of the instruction sample, while purple text indicates the target outputs optimized during fine-tuning. For detailed examples, refer to Figures \ref{['tab:snips-dst']} - \ref{['tab:sgd-sft-response']}.
• Figure 3: Error Analysis of Function-Calling Results. Illustrated performance comparison on function calling benchmarks API-Bank L1 (top) and BFCL V3 parallel function call (bottom). Results demonstrate CoALM's consistent performance compared to other baselines.
• Figure 4: SNIPS fine-tuning sample example.
• Figure 5: Hammer fine-tuning sample example.