ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems

TL;DR

ToolCritic proposes a diagnostic framework that identifies eight tool-use errors in multi-turn tool-augmented dialogues and delivers descriptive feedback to an LLM for on-the-fly revision. Trained on a synthetically augmented Schema-Guided Dialogue dataset, ToolCritic demonstrates up to a 13% boost in tool-calling success over baselines and validates the necessity of external feedback over self-correction. The work provides a granular error taxonomy, a synthetic data generation pipeline, and a feedback loop that enhances robustness of tool usage in real-world dialogue applications. While effective, the approach incurs higher inference and labeling costs and currently relies on a fixed tool set, leaving room for improved generalization to unseen tools and more efficient reasoning.

Abstract

Tool-augmented large language models (LLMs) are increasingly employed in real-world applications, but tool usage errors still hinder their reliability. We introduce ToolCritic, a diagnostic framework that evaluates and improves LLM behavior in multi-turn, tool-augmented dialogues. ToolCritic detects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinterpretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic's feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more robust LLM integration with external tools in real-world dialogue applications.

Figures (7)

• Figure 1: ToolCritic evaluates every assistant response in a conversation, whether a tool was called or not (this is not shown in the diagram due to space limitations). If ToolCritic detects an assistant error, it will produce a reasoning "thought" which will be fed back to the assistant. The assistant then will revise its own response based on the received feedback and produce an updated response. If ToolCritic does not detect a mistake, the conversation continues without interference. Note that while we show the Assistant "Initial Response" for clarity, in a real-world scenario, this response will be hidden from the user and only the revised response will be used.
• Figure 2: Each error category defines a specific mistake, or failure mode, that is common for LLMs when interacting with tools in multi-turn dialogue. This granularity ensures the resulting diagnostic model, ToolCritic, can provide very specific feedback for the assistant LLM.
• Figure 3: Evaluation results of ToolCritic on the test data $D^{\text{test}}$.
• Figure 4: Flowchart illustrating the evaluation process for each turn in the dialogue. ToolCritic evaluates the LLM's response, providing feedback for correction if an error is detected. Note that the LLM performs a single attempt at correction and the revised response is passed to the user without additional evaluation. This allows us to balance accuracy with inference and latency cost.
• Figure 5: "Error Profile" of Claude 3 Sonnet. The diagram shows the distribution of mistakes detected by ToolCritic when Claude 3 Sonnet serves as the assistant on the SGD test set.