The Biased Oracle: Assessing LLMs' Understandability and Empathy in Medical Diagnoses

TL;DR

The paper investigates how large language models communicate medical diagnoses by measuring understandability via readability metrics and empathy via an LLM-as-a-Judge against human judgments. It designs 156 prompts spanning diverse demographics and six scenarios across obesity, pancreatic cancer, Alzheimer's, and ischemic heart disease. The work finds that GPT-4o and Claude-3.7 produce explanations that vary with patient education and demographic group, often becoming overly complex, and exhibit affective empathy biases; cognitive empathy remains relatively stable. It also reveals biases in LLM-based empathy judgments and notable differences between model and human assessments, underscoring the need for calibrated, bias-aware deployment in patient communication.

Abstract

Large language models (LLMs) show promise for supporting clinicians in diagnostic communication by generating explanations and guidance for patients. Yet their ability to produce outputs that are both understandable and empathetic remains uncertain. We evaluate two leading LLMs on medical diagnostic scenarios, assessing understandability using readability metrics as a proxy and empathy through LLM-as-a-Judge ratings compared to human evaluations. The results indicate that LLMs adapt explanations to socio-demographic variables and patient conditions. However, they also generate overly complex content and display biased affective empathy, leading to uneven accessibility and support. These patterns underscore the need for systematic calibration to ensure equitable patient communication. The code and data are released: https://github.com/Jeffateth/Biased_Oracle

Figures (7)

• Figure 1: Evaluation framework for LLM-based medical diagnoses, assessing understandability (readability metrics) and (affective and cognitive) empathy (LLM vs. human judgment) across diverse demographic profiles.
• Figure 2: Understandability analysis of GPT and Claude outputs across readability metrics and demographics (a–d). Higher grade-level indices indicate greater complexity. Bars show Means; error bars denote $\pm$ 95% confidence intervals (CIs).
• Figure 3: Affective and cognitive empathy scores by (a) age group, (b) medical diagnosis, (c) education level, (d) geographical group, and (e) gender. Abbreviations (plots b,c): PanCan = Pancreatic cancer; CIHD = Chronic Ischemic Heart Disease; Obes = Obesity; Alz = Alzheimer's disease; HS = High school diploma or lower; Univ = University degree; Med = Medical degree. Legend abbreviations (all panels): C$\rightarrow$C = Claude response rated by Claude; C$\rightarrow$G = Claude response rated by GPT; G$\rightarrow$C = GPT response rated by Claude; G$\rightarrow$G = GPT response rated by GPT. Bars show Means; error bars denote 95% CIs. Panel (e) shows Female$-$Male mean differences (positive = higher scores for females).
• Figure 4: Human vs. LLM empathy ratings on GPT-generated responses across demographic groups. Bars show mean scores; error bars denote 95% CIs.
• Figure 5: Model bias vs. human rating variability on GPT-generated responses.