Measuring Stability Beyond Accuracy in Small Open-Source Medical Large Language Models for Pediatric Endocrinology

TL;DR

Measuring Stability Beyond Accuracy in Small Open-Source Medical LLMs for Pediatric Endocrinology evaluates six sub-10B open-source LLMs on Pediatric ESAP MCQs to move beyond traditional accuracy metrics. The authors implement a multidimensional framework that probes prompt sensitivity, output stability under stochastic decoding, self-bias in reasoning, and numerical reproducibility across CUDA environments. Results show HuatuoGPT-o1-8B achieves the highest accuracy but still exhibits self-bias, dependence on prompt structure, and limited clinical reasoning, while other models display considerable variability and artifacts. The study highlights that consistency in outputs does not imply correctness and argues for a broader diagnostic framework, including scenario-based evaluation and retrieval augmentation, to enable safer clinical deployment in low-resource settings.

Abstract

Small open-source medical large language models (LLMs) offer promising opportunities for low-resource deployment and broader accessibility. However, their evaluation is often limited to accuracy on medical multiple choice question (MCQ) benchmarks, and lacks evaluation of consistency, robustness, or reasoning behavior. We use MCQ coupled to human evaluation and clinical review to assess six small open-source medical LLMs (HuatuoGPT-o1 (Chen 2024), Diabetica-7B, Diabetica-o1 (Wei 2024), Meditron3-8B (Sallinen2025), MedFound-7B (Liu 2025), and ClinicaGPT-base-zh (Wang 2023)) in pediatric endocrinology. In deterministic settings, we examine the effect of prompt variation on models' output and self-assessment bias. In stochastic settings, we evaluate output variability and investigate the relationship between consistency and correctness. HuatuoGPT-o1-8B achieved the highest performance. The results show that high consistency across the model response is not an indicator of correctness, although HuatuoGPT-o1-8B showed the highest consistency rate. When tasked with selecting correct reasoning, both HuatuoGPT-o1-8B and Diabetica-o1 exhibit self-assessment bias and dependency on the order of the candidate explanations. Expert review of incorrect reasoning rationales identified a mix of clinically acceptable responses and clinical oversight. We further show that system-level perturbations, such as differences in CUDA builds, can yield statistically significant shifts in model output despite stable accuracy. This work demonstrates that small, semantically negligible prompt perturbations lead to divergent outputs, raising concerns about reproducibility of LLM-based evaluations and highlights the output variability under different stochastic regimes, emphasizing the need of a broader diagnostic framework to understand potential pitfalls in real-world clinical decision support scenarios.

Figures (11)

• Figure 1: Consistency and correctness in the stochastic setting. From left to right, representations for HuatuoGPT-o1-8B, Diabetica-o1, Diabetica-7B, and Meditron 3-8B. From top to bottom, results across different temperature setting with $T=0.3$ (blue frame), $T=0.6$ (gray frame), and $T=1.0$ (yellow frame). In each plot, the $y$-axis reports consistency as majority vote selected $\ge 7$ times, up to ten. Correctness varies from 0 to total number of runs ($=10)$, on the $x$-axis. The size and color intensity of the blob show the number of cases that fall into a specific category. Ideally, a perfect-scoring model would show all the density concentrated in a single blob at the top right corner of the plot. We excluded all ESAP items where the model lacks a majority class.
• Figure 2: Box-plot showing the distribution of correct responses across runs, at three temperatures (color codes as in Figure \ref{['fig:Figure1a']}). The solid horizontal line shows performance under deterministic condition for each model.
• Figure 3: HuatuoGPT-o1-8B and Diabetica-o1 selection of gold-standard explanation against their own reasoning under prompt A and prompt B. Each bar reports the percentage of times each model selected the gold standard explanation, regardless of the presented order (ESAP both in green), if only in one of the two positions (ESAP first in khaki or ESAP second in yellow), in neither (ESAP never in red). Cases where a model did not make a valid selection, either by failing to choose between the two explanations or by introducing a new and unrelated option are indicated as Hallucinate / None, in gray.
• Figure 4: Graphical representation of HuatuoGPT-o1-8B consistency and correctness at different temperature values.
• Figure 5: Graphical representation of Diabetica-o1 consistency and correctness at different temperature values.