The Reliability of LLMs for Medical Diagnosis: An Examination of Consistency, Manipulation, and Contextual Awareness

TL;DR

This work states that responsible clinical deployment in resource-scarce and trust-dependent environments, demands comprehensive reliability evaluation, which must go beyond accuracy to encompass diagnostic consistency, manipulation resilience and intelligent contextual integration, ensuring the safe and ethical application of LLMs for universal healthcare.

Abstract

Universal healthcare access is critically needed, especially in resource-limited settings. Large Language Models (LLMs) offer promise for democratizing healthcare with advanced diagnostics, but their reliability requires thorough evaluation, especially in trust-dependent environments. This study assesses LLMs' diagnostic reliability focusing on consistency, manipulation resilience, and contextual integration, crucial for safe and ethical use in universal healthcare. We evaluated leading LLMs using 52 patient cases, expanded into variants with demographic changes, symptom rewordings, and exam modifications, while keeping core diagnoses constant. Manipulation susceptibility was tested by inserting misleading narratives and irrelevant details. Contextual awareness was rvaluated by comparing diagnoses with and without patient history. We analyzed diagnostic change rates and response patterns across manipulations. LLMs showed perfect diagnostic consistency for identical data but significant manipulation susceptibility. Gemini had a 40% diagnosis change rate and ChatGPT 30% with irrelevant details. ChatGPT had a higher context influence rate (77.8% vs. Gemini's 55.6%), but both showed limited nuanced contextual integration, exhibiting anchoring bias by prioritizing salient data over context. LLMs' vulnerability to manipulation and limited contextual awareness pose challenges in clinical use. Unlike clinicians, they may overstate diagnostic certainty without validation. Safeguards and domain-specific designs are crucial for reliable healthcare applications. Broad clinical use without oversight is premature and risky. LLMs can enhance diagnostics with responsible use, but future research is needed to improve manipulation resistance and contextual understanding for safe healthcare democratization.

Figures (9)

• Figure 1: Dataset Construction Methodology
• Figure 2: Bar Chart Visualization of Baseline Scenario Consistency of LLMs
• Figure 7: Diagnosis Category Distribution per LLM - pie chart
• Figure 8: Distribution of Appropriateness Ratings by LLM (Physician Y)
• Figure 9: Distribution of Appropriateness Ratings by LLM (Physician X)