AgentEHR: Advancing Autonomous Clinical Decision-Making via Retrospective Summarization

TL;DR

AgentEHR targets realistic autonomous clinical decision-making within raw, high-noise EHR data, addressing the information-loss and reasoning-discontinuity limitations of prior summarization methods. It introduces RetroSum, a retrospective summarization framework paired with an evolving memory strategy to preserve long-horizon clinical reasoning while leveraging accumulated experiences. Empirical results across MIMIC-IV and MIMIC-III show RetroSum achieving up to 29.16% performance gains and up to 92.3% total-error reductions compared with competitive baselines, with robustness to distribution shifts and improved efficiency. The work establishes a new benchmark for EHR-based autonomous agents and highlights the importance of memory-augmented, history-aware reasoning for practical clinical applications.

Abstract

Large Language Models have demonstrated profound utility in the medical domain. However, their application to autonomous Electronic Health Records~(EHRs) navigation remains constrained by a reliance on curated inputs and simplified retrieval tasks. To bridge the gap between idealized experimental settings and realistic clinical environments, we present AgentEHR. This benchmark challenges agents to execute complex decision-making tasks, such as diagnosis and treatment planning, requiring long-range interactive reasoning directly within raw and high-noise databases. In tackling these tasks, we identify that existing summarization methods inevitably suffer from critical information loss and fractured reasoning continuity. To address this, we propose RetroSum, a novel framework that unifies a retrospective summarization mechanism with an evolving experience strategy. By dynamically re-evaluating interaction history, the retrospective mechanism prevents long-context information loss and ensures unbroken logical coherence. Additionally, the evolving strategy bridges the domain gap by retrieving accumulated experience from a memory bank. Extensive empirical evaluations demonstrate that RetroSum achieves performance gains of up to 29.16% over competitive baselines, while significantly decreasing total interaction errors by up to 92.3%.

Figures (12)

• Figure 1: Comparison between the previous EHR tasks and the proposed benchmark. Unlike previous EHRAgent task retrieving factual information explicitly present in the EHR (e.g., medication history), AgentEHR analyzes existing information to predict future clinical decisions, like diagnosis and treatment plans.
• Figure 2: Overview of RetroSum. RetroSum (right) addresses critical information loss and reasoning interruptions inherent in unidirectional methods like ReSum (left). By incorporating a retrospective mechanism to re-evaluate full interaction histories and an evolving mechanism to retrieve specialized strategies from memory, RetroSum ensures robust long-horizon clinical reasoning and correct decisions.
• Figure 3: Impact of summarization interval on agent performance. The retrospective mechanism is applied solely to the Summarizer (Sum-Only) or the Actor (Act-Only) across varying frequencies.
• Figure 4: Error statistics on the diagnoses task. The left panel displays the distribution of specific error types across unsuccessful and successful trajectories. The right panel compares the count of errors committed by different agent frameworks.
• Figure 5: Distribution of interaction turns across different agent methods.