Adaptive Reasoning and Acting in Medical Language Agents

TL;DR

An innovative large language model (LLM) agent framework for enhancing diagnostic accuracy in simulated clinical environments using the AgentClinic benchmark and enables doctor agents to iteratively refine their reasoning and actions following incorrect diagnoses, fostering improved decision-making over time.

Abstract

This paper presents an innovative large language model (LLM) agent framework for enhancing diagnostic accuracy in simulated clinical environments using the AgentClinic benchmark. The proposed automatic correction enables doctor agents to iteratively refine their reasoning and actions following incorrect diagnoses, fostering improved decision-making over time. Experiments show that the implementation of the adaptive LLM-based doctor agents achieve correct diagnoses through dynamic interactions with simulated patients. The evaluations highlight the capacity of autonomous agents to adapt and improve in complex medical scenarios. Future enhancements will focus on refining the algorithm and expanding its applicability across a wider range of tasks and different large language models.

Figures (5)

• Figure 1: An architecture towards autonomous agent. Created with BioRender.com.
• Figure 2: Medical Self-Adaptive Language Agent. Created with BioRender.com.
• Figure 3: The clinical consultation dialogue of the first case in the MedQA simulated clinical environment in schmidgall2024agentclinic from a GPT-4 achiam2023gpt doctor, patient, measurement, and moderator language agent. The doctor correctly diagnosed the patient with Myasthenia Gravis.
• Figure 4: The clinical consultation dialogue of the first case in the MedQA simulated clinical environment in schmidgall2024agentclinic from a GPT-3.5 brown2020language doctor language agent and GPT-4 patient, measurement, and moderator language agent. The doctor misdiagnosed the patient with Guillain-Barré Syndrome, but the patient actually has Myasthenia Gravis.
• Figure 5: The clinical consultation dialogue of the first case in the MedQA simulated clinical environment in schmidgall2024agentclinic from a GPT-3.5 brown2020language doctor language agent and GPT-4 patient, measurement, and moderator language agent using our proposed method. The doctor correctly diagnosed the patient with Myasthenia Gravis with 1 test and 12 questions, which is less than the doctor agent in Firgure \ref{['fig:GPT4Result1']} with 1 test and 19 questions.