MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis

TL;DR

This work identifies the Einstellung Effect as a critical failure mode in medical LLMs, where models rely on priors rather than patient-specific evidence in differential diagnosis. It introduces MedEinst, a counterfactual benchmark with 5,383 paired cases across 49 diseases, designed to quantify susceptibility via Bias Trap Rate. To mitigate this, the authors propose ECR-Agent, which couples Dynamic Causal Inference with Critic-Driven Graph & Memory Evolution to enforce evidence-based, causal reasoning aligned with Evidence-Based Medicine. Empirical results show frontier models exhibit high baseline accuracy but high bias, while ECR-Agent significantly improves both factual diagnosis and robustness by grounding decisions in discriminative evidence and evolving illness graphs. The work highlights that scaling alone does not guarantee robust medical reasoning and demonstrates a pathway toward trustworthy AI through structured causal reasoning and memory-augmented learning, with plans to release source code.

Abstract

Despite achieving high accuracy on medical benchmarks, LLMs exhibit the Einstellung Effect in clinical diagnosis--relying on statistical shortcuts rather than patient-specific evidence, causing misdiagnosis in atypical cases. Existing benchmarks fail to detect this critical failure mode. We introduce MedEinst, a counterfactual benchmark with 5,383 paired clinical cases across 49 diseases. Each pair contains a control case and a "trap" case with altered discriminative evidence that flips the diagnosis. We measure susceptibility via Bias Trap Rate--probability of misdiagnosing traps despite correctly diagnosing controls. Extensive Evaluation of 17 LLMs shows frontier models achieve high baseline accuracy but severe bias trap rates. Thus, we propose ECR-Agent, aligning LLM reasoning with Evidence-Based Medicine standard via two components: (1) Dynamic Causal Inference (DCI) performs structured reasoning through dual-pathway perception, dynamic causal graph reasoning across three levels (association, intervention, counterfactual), and evidence audit for final diagnosis; (2) Critic-Driven Graph and Memory Evolution (CGME) iteratively refines the system by storing validated reasoning paths in an exemplar base and consolidating disease-specific knowledge into evolving illness graphs. Source code is to be released.

Figures (8)

• Figure 1: (a) Example of Einstellung Effect (b) Distribution of failure modes under the Einstellung Effect across reasoning LLMs, including Blindness (missing key evidence), Underthinking (insufficient reasoning), and Overthinking (rationalizing incorrect priors).
• Figure 2: Data construction of MedEinst with four-stage process: (1) Data Filtering for hard candidates, (2) Narration Conversion to natural language, (3) Differential Features Rewrite for trap case generation, and (4) Inter-Model Verification for quality control.
• Figure 3: ECR-Agent, aligning LLM reasoning with Evidence-Based Medicine via two parts: (a) Dynamic Causal Inference (DCI) performs structured reasoning via dual-pathway perception, dynamic causal graph reasoning across three levels, and evidence audit for final diagnosis. (b) Critic-Driven Graph & Memory Evolution (CGME) iteratively refines the system by storing validated reasoning paths in an exemplar base and consolidating disease-specific knowledge into evolving illness graphs.
• Figure 4: Bias Trap Rate heatmap across diseases. The clustering indicates that models learn spurious correlations for common diseases (e.g., Pneumonia), leading to consistent bias.
• Figure 5: Baseline Accuracy vs. Bias Trap Rate.