Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications

TL;DR

The paper addresses the gap between strong benchmark performance of medical LLMs and their safe deployment in clinical workflows. It proposes a level-of-autonomy framework (L0–L3) that ties evaluation targets to the permitted actions at each level, shifting from score-based claims to risk-aware, evidence-backed assessments. By mapping existing benchmarks and metrics to autonomy levels and detailing level-specific evaluation focuses, it provides a practical blueprint for trustworthy clinical integration, highlighting challenges in grounding, calibration, attribution, and tool use. The work advocates for reporting with risk-coverage metrics, governance provenance, and human-in-the-loop safeguards to enable credible, safe, and scalable medical AI deployments across information, transformation, decision support, and agent-based workflows.

Abstract

Medical Large language models achieve strong scores on standard benchmarks; however, the transfer of those results to safe and reliable performance in clinical workflows remains a challenge. This survey reframes evaluation through a levels-of-autonomy lens (L0-L3), spanning informational tools, information transformation and aggregation, decision support, and supervised agents. We align existing benchmarks and metrics with the actions permitted at each level and their associated risks, making the evaluation targets explicit. This motivates a level-conditioned blueprint for selecting metrics, assembling evidence, and reporting claims, alongside directions that link evaluation to oversight. By centering autonomy, the survey moves the field beyond score-based claims toward credible, risk-aware evidence for real clinical use.

Figures (1)

• Figure 1: Overview of autonomy levels (L0--L3) for medical LLMs, showing for each level the scope, typical applications, an example question/task, and key challenges. The rightward arrow indicates increasing autonomy and risk.