Counterfactual Cultural Cues Reduce Medical QA Accuracy in LLMs: Identifier vs Context Effects

TL;DR

The paper tackles the problem of cultural bias in medical QA by introducing a counterfactual benchmark that expands $150$ MedQA items into $1650$ variants across three cultural groups using non-decisive identifiers, contextual cues, or both, plus a neutral control. It evaluates four LLMs under option-only and brief explanation prompts, demonstrating that cultural cues degrade diagnostic accuracy with the largest drops when identity and context co-occur, and that more than half of culturally grounded explanations lead to incorrect answers; these effects are statistically significant at $p<10^{-14}$. The authors show that neutral edits and single-cue perturbations are less harmful, while the combination of identity and context reliably induces errors, and they provide a human-validated rubric (inter-rater reliability $\kappa=0.76$) plus an LLM-based judge to analyze reasoning. The work contributes prompts and augmentations to evaluate and mitigate culturally induced diagnostic errors, highlighting the need for culturally aware calibration in medical language models to improve safety and trust in real-world clinical use.

Abstract

Engineering sustainable and equitable healthcare requires medical language models that do not change clinically correct diagnoses when presented with non-decisive cultural information. We introduce a counterfactual benchmark that expands 150 MedQA test items into 1650 variants by inserting culture-related (i) identifier tokens, (ii) contextual cues, or (iii) their combination for three groups (Indigenous Canadian, Middle-Eastern Muslim, Southeast Asian), plus a length-matched neutral control, where a clinician verified that the gold answer remains invariant in all variants. We evaluate GPT-5.2, Llama-3.1-8B, DeepSeek-R1, and MedGemma (4B/27B) under option-only and brief-explanation prompting. Across models, cultural cues significantly affect accuracy (Cochran's Q, $p<10^-14$), with the largest degradation when identifier and context co-occur (up to 3-7 percentage points under option-only prompting), while neutral edits produce smaller, non-systematic changes. A human-validated rubric ($κ=0.76$) applied via an LLM-as-judge shows that more than half of culturally grounded explanations end in an incorrect answer, linking culture-referential reasoning to diagnostic failure. We release prompts and augmentations to support evaluation and mitigation of culturally induced diagnostic errors.

Figures (5)

• Figure 1: Overview of the pipeline for evaluation of cultural bias in medical question answering. MedQA questions are augmented using identity-based, context-based, and identity + context methods across three cultural backgrounds (Indigenous Canadian, Middle-Eastern Muslim, and Southeast Asian). Augmented questions are answered by multiple large language models (LLaMA-3.1, GPT-5.2, MedGemma-4B/27B, and DeepSeek-R1) under option-only and brief-explanation prompting. Model outputs are analyzed using statistical testing (Cochran’s Q) and an LLM-as-judge framework to detect culturally-induced errors. The code and data of this pipeline are available at https://github.com/HIVE-UofT/Evaluatiog-Cultural-Clues-Medical-LLMs.
• Figure 2: Culturally augmented MedQA questions. Each original question is transformed into identifier-only (Teal), context-only (Sage), and combined (Lavender) variants. Bold text shows the inserted cultural identifiers or contextual cues in the clinical scenario.
• Figure 3: Model robustness across cultural scenarios. Bar heights represent absolute accuracy, and error bars denote bootstrapped 95% confidence intervals ($n=2000$). The overlap between the Original and Neutral conditions across all models shows that prompt length alone does not reduce performance. The Identity + Context settings show statistically significant accuracy declines for DeepSeek-R1 and MedGemma-4B, with larger effects in Indigenous contexts.
• Figure 4: Component analysis of culturally induced errors. Bars show the mean percentage-point drop in accuracy relative to the baseline. For most models, the Identity Only condition produces a drop similar to Identity + Context, whereas Context Only has a smaller effect.
• Figure 5: Forest plot of model sensitivity to cultural contexts. Points represent the effect size, defined as the mean accuracy difference in percentage points between Identity + Context and the original baseline. The right-hand column reports point estimates with bootstrapped 95% confidence intervals. Bolded values indicate statistically significant bias because the confidence interval does not include 0. DeepSeek-R1 shows the largest sensitivity to Indigenous contexts, whereas MedGemma-27B remains robust across groups.