Shallow Robustness, Deep Vulnerabilities: Multi-Turn Evaluation of Medical LLMs

TL;DR

This work reveals a sharp divide between shallow and deep robustness in medical LLMs. It introduces MedQA-Followup, a framework and dataset for evaluating multi-turn vulnerabilities across 1,273 MedQA questions, distinguishing indirect contextual framing from direct suggestions. Empirical results show that while models handle single-turn perturbations reasonably, multi-turn and context-driven manipulations can drastically erode accuracy, with substantial model-to-model differences and some signs of recovery under certain compounding conditions. The findings underscore the need for multi-turn evaluation, domain-aware deployment strategies, and safeguards to ensure safe, reliable clinical use of medical LLMs.

Abstract

Large language models (LLMs) are rapidly transitioning into medical clinical use, yet their reliability under realistic, multi-turn interactions remains poorly understood. Existing evaluation frameworks typically assess single-turn question answering under idealized conditions, overlooking the complexities of medical consultations where conflicting input, misleading context, and authority influence are common. We introduce MedQA-Followup, a framework for systematically evaluating multi-turn robustness in medical question answering. Our approach distinguishes between shallow robustness (resisting misleading initial context) and deep robustness (maintaining accuracy when answers are challenged across turns), while also introducing an indirect-direct axis that separates contextual framing (indirect) from explicit suggestion (direct). Using controlled interventions on the MedQA dataset, we evaluate five state-of-the-art LLMs and find that while models perform reasonably well under shallow perturbations, they exhibit severe vulnerabilities in multi-turn settings, with accuracy dropping from 91.2% to as low as 13.5% for Claude Sonnet 4. Counterintuitively, indirect, context-based interventions are often more harmful than direct suggestions, yielding larger accuracy drops across models and exposing a significant vulnerability for clinical deployment. Further compounding analyses reveal model differences, with some showing additional performance drops under repeated interventions while others partially recovering or even improving. These findings highlight multi-turn robustness as a critical but underexplored dimension for safe and reliable deployment of medical LLMs.

Figures (9)

• Figure 1: Examples of Interventions in MedQA: illustration of different single-turn and multi-turn interventions (with techniques in parenthesis in gray) applied to a MedQA example. Turns with white backgrounds represent user inputs, while the gray background corresponds to the assistant’s response. Text in light blue indicates the original Q&A from the dataset; text in purple denotes static context added uniformly across all interventions; and text in red represents content generated specifically for the given question, options, and correct answer.
• Figure 2: Robustness in Medical Q&A: average relative percentage change with respect to baseline accuracy across different categories of single-turn and multi-turn (follow-up) interventions.
• Figure 3: context length ablation: model accuracy as a function of the context length for Claude Sonnet 4, GPT-4.1, and MedGemma 27B across the different techniques supported. Qualitative examples are shown in Appendix \ref{['app:qualitative-examples-context-length-ablations']}
• Figure 4: Compounding intervention effects: relative drop of the compounded intervention compared to the baseline drops of the separate follow-ups, split into one-turn compounding (top) and two-turn compounding (bottom).
• Figure 5: Expected vs. observed compounding: relative drop in accuracy from baseline that could be expected by compounding multiple effects (i.e., taking the worst case error) plotted against the actually observed relative drop in accuracy. The area in purple corresponds to sub-additive effects (where the effect was smaller than expected) whereas in pink we show super-additive ones.