From Failure to Mastery: Generating Hard Samples for Tool-use Agents

TL;DR

HardGen presents a failure-driven pipeline to generate hard, verifiable tool-use data for LLMs. It builds a dynamic API Graph from model failures, samples hard traces, evolves advanced tools conditioned on those traces, and refines Chain-of-Thought reasoning through a closed-loop Reasoner-Verifier mechanism. The dataset comprises 27,000 trajectories over 2,095 APIs, enabling a 4B-Qwen-variant model trained with HardGen data to achieve state-of-the-art results on BFCLv3 and strong generalization to BFCLv4 and other benchmarks. The approach demonstrates robust gains through hard-query generation, advanced tool abstractions, and feedback-guided CoT refinement, and the authors commit to open-sourcing code, models, and data to spur future research.

Abstract

The advancement of LLM agents with tool-use capabilities requires diverse and complex training corpora. Existing data generation methods, which predominantly follow a paradigm of random sampling and shallow generation, often yield simple and homogeneous trajectories that fail to capture complex, implicit logical dependencies. To bridge this gap, we introduce HardGen, an automatic agentic pipeline designed to generate hard tool-use training samples with verifiable reasoning. Firstly, HardGen establishes a dynamic API Graph built upon agent failure cases, from which it samples to synthesize hard traces. Secondly, these traces serve as conditional priors to guide the instantiation of modular, abstract advanced tools, which are subsequently leveraged to formulate hard queries. Finally, the advanced tools and hard queries enable the generation of verifiable complex Chain-of-Thought (CoT), with a closed-loop evaluation feedback steering the continuous refinement of the process. Extensive evaluations demonstrate that a 4B parameter model trained with our curated dataset achieves superior performance compared to several leading open-source and closed-source competitors (e.g., GPT-5.2, Gemini-3-Pro and Claude-Opus-4.5). Our code, models, and dataset will be open-sourced to facilitate future research.

Figures (15)

• Figure 1: Performance comparison on the BFCLv3Leaderboardpatilberkeley. A Qwen3-4B yang2025qwen3 model trained with our curated dataset, denoted as HardGen-4B-RL, consistently outperforms leading open-source and closed-source models.
• Figure 2: Overview of the HardGen framework. The pipeline operates in three phases: I) Failure-inspired Hard Trace Sampling to identify error-prone tool dependencies and construct hard tool traces; II) Trace-conditioned Tool Evolution to synthesize advanced tools and hard queries based on the constructed hard traces; and III) Feedback-guided CoT Refinement to verify and optimize reasoning chains through a closed-loop mechanism.
• Figure 3: Statistics and distribution of the generated dataset. (Top) Histogram of tool calls per trajectory (left) and key dataset metrics (right). (Bottom) Distribution of API domains (left), the proportion of single-turn and multi-turn trajectories (right).
• Figure 4: Additional evaluations. (a) (b) Comparison of different models on APIBank and ACEBench. (c) Scaling trends of HardGen-SFT and HardGen-RL on BFCLv3 across model parameters.
• Figure 5: Analysis of generator model selection. Correctness rates of function call synthesis at different numbers of attempts ($K$) for three candidate generator models.