Uncovering Latent Bias in LLM-Based Emergency Department Triage Through Proxy Variables
Ethan Zhang
TL;DR
This paper addresses hidden biases in LLM-driven emergency department triage by introducing a proxy-variable framework that uses 32 patient-level proxies with paired positive/negative qualifiers to probe shifts in Emergency Severity Index ($ESI$) predictions. By evaluating both open (MIMIC‑IV‑ED Demo) and restricted datasets, the authors reveal polarity-dependent and polarity-independent biases arising from non-clinical signals, suggesting LLMs may overreact to contextual tokens rather than true patient acuity. They provide a reproducible methodology for quantifying acuity shifts, demonstrating how social determinants like race-linked proxy signals (e.g., ambulance use) can propagate discrimination in triage outputs. The findings emphasize the need for safer deployment and targeted bias mitigation in clinical AI, offering a practical evaluative framework for ongoing fairness auditing in high-stakes settings.
Abstract
Recent advances in large language models (LLMs) have enabled their integration into clinical decision-making; however, hidden biases against patients across racial, social, economic, and clinical backgrounds persist. In this study, we investigate bias in LLM-based medical AI systems applied to emergency department (ED) triage. We employ 32 patient-level proxy variables, each represented by paired positive and negative qualifiers, and evaluate their effects using both public (MIMIC-IV-ED Demo, MIMIC-IV Demo) and restricted-access credentialed (MIMIC-IV-ED and MIMIC-IV) datasets as appropriate~\cite{mimiciv_ed_demo,mimiciv_ed,mimiciv}. Our results reveal discriminatory behavior mediated through proxy variables in ED triage scenarios, as well as a systematic tendency for LLMs to modify perceived patient severity when specific tokens appear in the input context, regardless of whether they are framed positively or negatively. These findings indicate that AI systems is still imperfectly trained on noisy, sometimes non-causal signals that do not reliably reflect true patient acuity. Consequently, more needs to be done to ensure the safe and responsible deployment of AI technologies in clinical settings.