An Agentic AI Framework for Training General Practitioner Student Skills

TL;DR

The paper tackles scalable training of general practitioner skills using an agentic AI framework for virtual simulated patients (VSPs). It introduces a three-role architecture (scenario generator, VSP conversational agent, and standards-based critic) with evidence-based vignette grounding, controlled persona via Big Five traits, and automated feedback aligned to the Master Interview Rating Scale and clinical guidelines. An in-person evaluation with 14 medical students reports realistic dialogues, calibrated difficulty, stable personality signaling, and highly actionable feedback, along with strong usability. The results support a practical, extensible pattern for dependable VSP training tools and potential applicability to broader medical education tasks.

Abstract

Advancements in large language models offer strong potential for enhancing virtual simulated patients (VSPs) in medical education by providing scalable alternatives to resource-intensive traditional methods. However, current VSPs often struggle with medical accuracy, consistent roleplaying, scenario generation for VSP use, and educationally structured feedback. We introduce an agentic framework for training general practitioner student skills that unifies (i) configurable, evidence-based vignette generation, (ii) controlled persona-driven patient dialogue with optional retrieval grounding, and (iii) standards-based assessment and feedback for both communication and clinical reasoning. We instantiate the framework in an interactive spoken consultation setting and evaluate it with medical students ($\mathbf{N{=}14}$). Participants reported realistic and vignette-faithful dialogue, appropriate difficulty calibration, a stable personality signal, and highly useful example-rich feedback, alongside excellent overall usability. These results support agentic separation of scenario control, interaction control, and standards-based assessment as a practical pattern for building dependable and pedagogically valuable VSP training tools.

Figures (8)

• Figure 1: Core schema of the four main contributions of our framework: clinical vignette generation, a three-step VSP generation method, personality customization, and automated feedback generation
• Figure 2: Component diagram with the core contributions from Fig. \ref{['fig:maincontributions']} to the other components in the framework.
• Figure 3: A Furhat robotic head embodying the speech-to-text and text-to-speech of the VSP agent.
• Figure 4: Multi-step pre-processing and RAG steps of the VSP agent before response generation. The corresponding prompt can be found in the Supplementary Material.
• Figure 5: Screenshot of the client dashboard. The left side contains the conversation log, the avatar controls and the immediate feedback. The right side contains the VSP configuration parameters, the patient generation launch button, and the generation log button.