DispatchMAS: Fusing taxonomy and artificial intelligence agents for emergency medical services

TL;DR

The paper addresses the challenge of high-stakes EMS dispatch under uncertainty by proposing DispatchMAS, a taxonomy-grounded, LLM-powered multi-agent system that autonomously simulates caller–dispatcher interactions. It constructs a 32-chief-complaint taxonomy, six caller identities from MIMIC-III, and a six-phase call protocol, then grounds two interacting agents via a fact commons and a RAG-informed prompt framework within AutoGen. Hybrid evaluation—combining physician assessments (Guidance Efficacy and Dispatch Effectiveness) with automated linguistic metrics—demonstrates high clinical plausibility, good operational performance, and strong communication quality (AC1 > 0.70; 94% correct agent contact; 91% advisor provision; Flesch ~80.9; 60% polite). The results support using DispatchMAS for dispatcher training and protocol evaluation, and outline a path for safe integration of AI agents into emergency response with future real-time decision support and broader, multilingual applicability.

Abstract

Objective: Emergency medical dispatch (EMD) is a high-stakes process challenged by caller distress, ambiguity, and cognitive load. Large Language Models (LLMs) and Multi-Agent Systems (MAS) offer opportunities to augment dispatchers. This study aimed to develop and evaluate a taxonomy-grounded, LLM-powered multi-agent system for simulating realistic EMD scenarios. Methods: We constructed a clinical taxonomy (32 chief complaints, 6 caller identities from MIMIC-III) and a six-phase call protocol. Using this framework, we developed an AutoGen-based MAS with Caller and Dispatcher Agents. The system grounds interactions in a fact commons to ensure clinical plausibility and mitigate misinformation. We used a hybrid evaluation framework: four physicians assessed 100 simulated cases for "Guidance Efficacy" and "Dispatch Effectiveness," supplemented by automated linguistic analysis (sentiment, readability, politeness). Results: Human evaluation, with substantial inter-rater agreement (Gwe's AC1 > 0.70), confirmed the system's high performance. It demonstrated excellent Dispatch Effectiveness (e.g., 94 % contacting the correct potential other agents) and Guidance Efficacy (advice provided in 91 % of cases), both rated highly by physicians. Algorithmic metrics corroborated these findings, indicating a predominantly neutral affective profile (73.7 % neutral sentiment; 90.4 % neutral emotion), high readability (Flesch 80.9), and a consistently polite style (60.0 % polite; 0 % impolite). Conclusion: Our taxonomy-grounded MAS simulates diverse, clinically plausible dispatch scenarios with high fidelity. Findings support its use for dispatcher training, protocol evaluation, and as a foundation for real-time decision support. This work outlines a pathway for safely integrating advanced AI agents into emergency response workflows.

Figures (5)

• Figure 1: Overview of the three-phase methodology: (a) Taxonomy & Fact Commons, (b) Agentic System, and (c) Human–algorithm Hybrid Evaluation.
• Figure 2: Schematic overview of the LLM-based multi-agent emergency medical dispatch (EMD) simulation system. Patient information derived from MIMIC-III electronic health records (EHR) seeds realistic caller backgrounds and identities. Knowledge from the Emergency Medical Dispatch National Standard Curriculum—including chief complaint taxonomy, handling protocols, and call-handling phases—grounds both caller and dispatcher agents. The pipeline illustrates caller scenario generation and identity selection, dispatcher–caller interactions with turn-level chief-complaint classification and protocol retrieval, optional escalation to auxiliary agents, and delivery of pre-arrival instructions, as exemplified by the representative call transcript.
• Figure 3: Evaluation results in (a) dispatch effectiveness and (b) guidance efficacy. Bars represent the percentage distribution of binary (Yes/No) and ordinal (1–5) responses.
• Figure 4: Algorithmic evaluation comparing dispatcher and caller dialogue profiles across four key metrics: (a) sentiment distribution, (b) emotion classification, (c) readability based on Flesch Reading Ease scores, and (d) politeness levels.
• Figure 5: Post-hoc analysis of information collection and response dynamics. (a) The Information Collection Efficiency Score (1-5) is highest during the "Assessment" phase of the call, aligning with the core task of that period. (b) The trace of collection completeness over simulated time shows that the system gathers information most rapidly for "Life-Critical Events" (green line), demonstrating effective prioritization. (c) The average simulated response time is fastest for "Life-Critical Events" (1.8s), confirming the system's ability to adapt its conversational strategy to the urgency of the situation.