From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations
Benlu Wang, Iris Xia, Yifan Zhang, Junda Wang, Feiyun Ouyang, Shuo Han, Arman Cohan, Hong Yu, Zonghai Yao
TL;DR
The paper tackles the problem that clinical calculation benchmarks overemphasize final numeric accuracy and miss intermediate reasoning failures. It introduces a stepwise evaluation framework and an LLM-aided error attribution system, paired with the MedRaC modular pipeline that grounds formulas via Formula RAG and uses Python execution to reduce arithmetic errors, all without model fine-tuning. On MedCalc-Bench, these methods reveal previously hidden weaknesses and demonstrate substantial accuracy gains, especially for calculation-heavy tasks, while aligning with expert clinical judgments. The work proposes a methodological shift toward domain-grounded, explainable evaluation to improve trustworthiness and safety of LLM-based medical calculation tools in real-world applications.
Abstract
Large language models (LLMs) have demonstrated promising performance on medical benchmarks; however, their ability to perform medical calculations, a crucial aspect of clinical decision-making, remains underexplored and poorly evaluated. Existing benchmarks often assess only the final answer with a wide numerical tolerance, overlooking systematic reasoning failures and potentially causing serious clinical misjudgments. In this work, we revisit medical calculation evaluation with a stronger focus on clinical trustworthiness. First, we clean and restructure the MedCalc-Bench dataset and propose a new step-by-step evaluation pipeline that independently assesses formula selection, entity extraction, and arithmetic computation. Under this granular framework, the accuracy of GPT-4o drops from 62.7% to 43.6%, revealing errors masked by prior evaluations. Second, we introduce an automatic error analysis framework that generates structured attribution for each failure mode. Human evaluation confirms its alignment with expert judgment, enabling scalable and explainable diagnostics. Finally, we propose a modular agentic pipeline, MedRaC, that combines retrieval-augmented generation and Python-based code execution. Without any fine-tuning, MedRaC improves the accuracy of different LLMs from 16.35% up to 53.19%. Our work highlights the limitations of current benchmark practices and proposes a more clinically faithful methodology. By enabling transparent and transferable reasoning evaluation, we move closer to making LLM-based systems trustworthy for real-world medical applications.