From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models

TL;DR

Tool-augmented LLMs can leverage external tools like the Code Interpreter, but tool use can induce Tool-Induced Myopia (TIM), a reasoning degradation where the model leans on computations rather than internal deduction. To surface TIM, the authors introduce PyMath, a 1,679-problem benchmark, and a four-dimensional evaluation suite that measures outcome and reasoning quality beyond final accuracy. Across seven frontier models, TaLMs achieve higher final-answer accuracy yet exhibit degraded reasoning and increased TIM prevalence as tool usage rises, shifting errors from arithmetic to global reasoning failures. They propose prompting-based and DPO-based alignment mitigations, which improve both final accuracy and reasoning depth, highlighting safer integration of tools for mathematical problem solving.

Abstract

Tool-augmented Language Models (TaLMs) can invoke external tools to solve problems beyond their parametric capacity. However, it remains unclear whether these tool-enabled gains reflect trustworthy reasoning. Focusing on the Code Interpreter tool, we show that even when tools are selected and executed correctly, TaLMs treat tool outputs as substitutes for reasoning, producing solutions that appear correct but lack coherent justification. We term this failure mode Tool-Induced Myopia (TIM), and study it using PYMATH, a benchmark of 1,679 competition-level mathematical problems for which Python code is helpful but not sufficient. We further develop a multi-dimensional evaluation suite to quantify reasoning degradation in TaLMs relative to their non-tool counterparts. Our findings reveal that while TaLMs achieve up to a 19.3 percentage point gain in final-answer accuracy, their reasoning behavior consistently deteriorates (e.g., non-tool LLMs win up to 41.5% more often in pairwise comparisons of the reasoning process). This degradation intensifies with tool use; the more frequently a model invokes tools, the less coherent its reasoning becomes. Moreover, tool use shifts errors from arithmetic mistakes toward global reasoning failures (logic, assumption, creativity); with TIM present in ~55% of high-risk cases. Finally, we propose a preference-optimization-based framework that realigns TaLMs to use tools as assistive evidence, improving both final-answer accuracy and reasoning depth under tool use. Codes and data are available at: https://github.com/megagonlabs/TIM.

Figures (5)

• Figure 1: Comparison of Base LLM and Tool-augmented LLM (TaLM) reasoning. The Base LLM (top) derives the solution through step-by-step mathematical reasoning, while the TaLM (bottom) relies on empirical checks and multiple tool calls to search for the minimum, a failure mode characteristic of Tool-Induced Myopia (TIM).
• Figure 2: Reasoning behavior vs. tool usage. Metrics are computed over bins defined by the number of tool calls in model solution: {0--3, 4--7, 8--11, 12+}. (a) Win Rate (higher is better) shows that the Base model more frequently outperforms TaLM as tool usage increases, indicating a widening gap at higher call counts. (b) Miss Rate (higher is worse) generally rises with additional tool calls, indicating more missing steps in tool-dependent trajectories. (c) PRM Accuracy (higher is better) typically declines as the number of calls grows.
• Figure 3: Correlation between code complexity metrics and Miss Rate across TaLMs. (a) Pearson correlation coefficients for Line of Code and Cyclomatic Complexity with Miss Rate. (b) Corresponding p-values with significance thresholds at p=0.05 (red) and p=0.10 (orange). Asterisks denote marginal significance (*p<0.10, **p<0.05). No statistically significant correlations are found, suggesting that TIM is not driven by code complexity.
• Figure 4: Change in reasoning error rates after tool use ($\Delta = \text{TaLM} - \text{Base}$). Positive values indicate that an error type becomes more frequent when the model has access to Code Interpreter tool.
• Figure 5: Code Interpreter invocation rates across TaLMs. Thinking models (GPT-5, o4-mini, Gemini-2.5-Flash) use the tool on an average of 50% more problems than non-thinking models.