Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors

TL;DR

Fission-GRPO tackles the brittleness of small language models in multi-turn tool use by turning execution errors into dense, on-policy corrective supervision. It introduces a three-stage loop that (i) trains a base policy with standard GRPO exploration, (ii) uses a learned Error Simulator to produce diagnostic feedback for failed trajectories, and (iii) performs multiplicative recovery updates by resampling new rollouts conditioned on augmented error contexts. Empirically, it delivers state-of-the-art results on BFCL v4 Multi-Turn across Qwen3-1.7B, 4B, and 8B, with a 5.7% absolute improvement in error recovery and a 4% overall accuracy gain for Qwen3-8B, matching or surpassing specialized tools. The approach reduces error-distribution mismatch by aligning training with current on-policy error modes and shows potential for generalization to other iterative reasoning tasks such as debugging and structured search.

Abstract

Large language models (LLMs) can call tools effectively, yet they remain brittle in multi-turn execution: following a tool call error, smaller models often degenerate into repetitive invalid re-invocations, failing to interpret error feedback and self-correct. This brittleness hinders reliable real-world deployment, where the execution errors are inherently inevitable during tool interaction procedures. We identify a key limitation of current approaches: standard reinforcement learning (RL) treats errors as sparse negative rewards, providing no guidance on how to recover, while pre-collected synthetic error-correction datasets suffer from distribution mismatch with the model's on-policy error modes. To bridge this gap, we propose Fission-GRPO, a framework that converts execution errors into corrective supervision within the RL training loop. Our core mechanism fissions each failed trajectory into a new training instance by augmenting it with diagnostic feedback from a finetuned Error Simulator, then resampling recovery rollouts on-policy. This enables the model to learn from the precise errors it makes during exploration, rather than from static, pre-collected error cases. On the BFCL v4 Multi-Turn, Fission-GRPO improves the error recovery rate of Qwen3-8B by 5.7% absolute, crucially, yielding a 4% overall accuracy gain (42.75% to 46.75%) over GRPO and outperforming specialized tool-use agents.

Figures (6)

• Figure 1: Error recovery is a key bottleneck for smaller tool-using models in multi-turn execution. (a) shows a representative hallucinated retry loop after an API error, while (b) reports recovery rates on BFCL v4 across model scales.
• Figure 2: Overview of the Fission-GRPO Framework. The framework operates in three stages: (1) Standard Exploration, utilizing GRPO to optimize policy $\pi_\theta$ on the query distribution $\mathcal{D}$; (2) Error Identification & Synthesis, where a simulator $\mathcal{S}_\phi$ generates diagnostic feedback for filtered error trajectories; and (3) Fission-based Update, where corrective samples trigger a multiplicative resampling process (factor $G'$) to align the policy with recovery paths.
• Figure 3: Performance decomposition on BFCL v4 Multi-Turn (Qwen3-8B).
• Figure 4: Multi-turn performance across different correction trigger intervals ($N$) on BFCL v4 Multi-Turn.
• Figure 6: Detailed visualization of Multi-turn Error Recovery. Comparisons of trajectories generated by Qwen3-8B under different training regimes. The Base model collapses due to immediate state loss; the GRPO model suffers from latent state mismatch leading to hallucination in later turns; Fission-GRPO overcomes this by employing diagnostic tools (find) to actively resolve state uncertainties.