Linguistic and Argument Diversity in Synthetic Data for Function-Calling Agents

TL;DR

The paper addresses the challenge of training function-calling agents by emphasizing linguistic diversity and argument-value diversity, beyond merely expanding the set of invoked functions. It introduces a general-purpose diversity-generation component that optimizes broad metrics without relying on hand-crafted prompts or taxonomies, enabling domain adaptation. Through intrinsic and extrinsic evaluations against SoTA baselines ToolACE and APIGen, the approach achieves substantially higher diversity while maintaining correctness, and yields notable out-of-distribution improvements on BFCL. A concrete demonstration shows that fine-tuning a LoRA-based LLM on the synthetic data improves cross-dataset robustness across BFCL, APIGen, and ToolAce benchmarks, highlighting practical impact for real-world tool-use systems.

Abstract

The construction of function calling agents has emerged as a promising avenue for extending model capabilities. A major challenge for this task is obtaining high quality diverse data for training. Prior work emphasizes diversity in functions, invocation patterns, and interaction turns, yet linguistic diversity of requests and coverage of arguments (e.g., \texttt{city\_name}, \texttt{stock\_ticker}) remain underexplored. We propose a method that generates synthetic datasets via optimizing general-purpose diversity metrics across both queries and arguments, without relying on hand-crafted rules or taxonomies, making it robust to different usecases. We demonstrate the effectiveness of our technique via both intrinsic and extrinsic testing, comparing it to SoTA data generation methods. We show a superiority over baselines in terms of diversity, while keeping comparable correctness. Additionally, when used as a training set, the model resulting from our dataset exhibits superior performance compared to analogous models based on the baseline data generation methods in out-of-distribution performance. In particular, we achieve an $7.4\%$ increase in accuracy on the BFCL benchmark compared to similar counterparts.

Figures (2)

• Figure 1: Currency word-cloud for ToolAce (left) and our method (right).
• Figure 2: Comparison of manual quality annotations across four metrics for 100 randomly sampled examples from ToolAce, APIGen, and our dataset.