Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty

TL;DR

MedAbstain addresses clinical uncertainty by unifying conformal prediction with explicit abstention for medical MCQA. The benchmark introduces abstention variants (A, NAP, AP) and leverages CP to quantify uncertainty (APS, LAC) under zero-shot and few-shot prompts, including Chain-of-Thought and thinking modes, across open- and closed-source models. Key findings show a strong link between abstention awareness and increased model uncertainty, with explicit abstention having a larger impact on reliability than perturbations alone, and CP offering robust uncertainty quantification across models. The work demonstrates substantial potential for safer deployment in high-stakes medical settings while highlighting gaps in calibration for proprietary models and the need for human oversight in clinical decision support, supported by human judgements on perturbations and abstention appropriateness.

Abstract

Current evaluation of large language models (LLMs) overwhelmingly prioritizes accuracy; however, in real-world and safety-critical applications, the ability to abstain when uncertain is equally vital for trustworthy deployment. We introduce MedAbstain, a unified benchmark and evaluation protocol for abstention in medical multiple-choice question answering (MCQA) -- a discrete-choice setting that generalizes to agentic action selection -- integrating conformal prediction, adversarial question perturbations, and explicit abstention options. Our systematic evaluation of both open- and closed-source LLMs reveals that even state-of-the-art, high-accuracy models often fail to abstain with uncertain. Notably, providing explicit abstention options consistently increases model uncertainty and safer abstention, far more than input perturbations, while scaling model size or advanced prompting brings little improvement. These findings highlight the central role of abstention mechanisms for trustworthy LLM deployment and offer practical guidance for improving safety in high-stakes applications.

Figures (15)

• Figure 1: Overview of the MedAbstain evaluation pipeline. For each question, we begin with the original NA variant ① and its options. An abstention option ② is inserted at a random position, forming the A variant. For perturbed variants, a SoTA LLM (gpt-4.1-mini) identifies and removes critical information (③) from the original question, making it more ambiguous; this yields the NAP variant. Adding the abstention option yields the AP Variant. For each variant, the model predicts the answer, and we extract logits/logprobs, shown as output bar charts. The highlighted purple bar shows the abstention probability: with the complete question, the abstention option increases model confusion; for the AP variant, uncertainty remains high, but the model favors abstention. The quantile threshold $\hat{q}$ is set using 30% of the data as a calibration set and applied to the remaining 70%. This process is repeated for both open- and closed-source LLM families.
• Figure 2: Amboss: Comparing performance across MedAbstain variants. The abstention option has the highest impact on the model's uncertainty as can be observed from A and AP variants.
• Figure 3: MedQA: Comparing performance across MedAbstain variants. The abstention option has the highest impact on the model's uncertainty as can be observed from A and AP variants.
• Figure 4: Amboss: Zeroshot vs Fewshot settings comparison. Few‑shot gives modest accuracy gains while slightly tightening LAC ($APS \approx 0$), especially under CoT. ots = median $\Delta x$, bars = IQR
• Figure 5: MedQA: Zeroshot vs Fewshot settings comparison. Few-shot improves accuracy marginally with the highest in A CoT—and often shrinks LAC under CoT ($\text{APS} \approx 0$); dots $=$ median $\Delta x$, bars $=$ IQR.