How Clinicians Think and What AI Can Learn From It

TL;DR

The paper reframes clinical AI from a sole focus on predictive accuracy to robust decision-making under uncertainty. It argues that medicine operates with ordinal, non-compensatory reasoning and that fast-and-frugal heuristics capture the actual decision structure better than dense probabilistic scoring in many settings. By adopting robust ordinal rules—via credal sets, dominance, $\epsilon$-dominance, and selective tie-breaking—the proposed architecture aims to reduce decision instability, harm from information overload, and pathway cascades while preserving clinical value. A practical blueprint is offered: separate rich belief representations from a conservative decision layer that outputs sets of admissible actions and uses selective computation when it can meaningfully alter outcomes. The framework provides testable predictions and evaluation strategies focused on net benefit, decision stability, and real-world workflow impact, with implications for safer, more patient-aligned AI adoption in clinical care.

Abstract

Most clinical AI systems operate as prediction engines -- producing labels or risk scores -- yet real clinical reasoning is a time-bounded, sequential control problem under uncertainty. Clinicians interleave information gathering with irreversible actions, guided by regret, constraints and patient values. We argue that the dominant computational substrate of clinician reasoning is not cardinal optimization but ordinal, non-compensatory decision-making: Clinicians frequently rely on fast-and-frugal, lexicographic heuristics (e.g., fast-and-frugal trees) that stop early after checking a small, fixed sequence of cues. We provide a normative rationale for why such algorithms are not merely bounded rationality shortcuts, but can be epistemically preferred in medicine. First, many clinical trade-offs are constructed through human judgment and are only weakly measurable on absolute scales; without strong measurement axioms, only orderings are invariant, motivating an ordinal-by-default stance. Second, preference and signal elicitation are structurally crude: The mapping from truth $\to$ perception $\to$ inference $\to$ recorded variables introduces layered noise, leaving a persistent uncertainty floor. When this 'crudeness' overwhelms the decision margin, plug-in expected-utility optimization becomes brittle (high flip probability under small perturbations), whereas robust dominance/filtering rules ($ε$-dominance, maximin) stabilize decisions.Finally, we outline a clinician-aligned AI blueprint: Use rich models for beliefs and trajectories, but choose actions through robust ordinal rules; treat heuristics as the low-dimensional special case; and deploy AI as 'selective complexity' -- invoked mainly for tie-breaking when decisions are fragile and information has positive expected impact.