CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation

TL;DR

CXRAgent introduces a director-guided, multi-stage reasoning framework for chest X-ray interpretation that coordinates diverse CXR analysis tools, validates outputs with visual evidence via an EDV module, and dynamically assembles expert teams for collaborative decision-making. The method combines iterative tool invocation (ReAct-style), evidence-grounded validation, flexible diagnostic planning with four collaboration modes, and adaptive team recruitment to handle complex and ambiguous cases. Experimental results show substantial gains across CheXbench, Medical-CXR-VQA, and MIMIC-CXR report generation, with strong robustness to tool conflicts and improved interpretability through visual grounding. The work demonstrates practical potential for clinically reliable CXR analysis by bridging tool diversity, evidence-based reasoning, and MDT-like collaboration.

Abstract

Chest X-ray (CXR) plays a pivotal role in clinical diagnosis, and a variety of task-specific and foundation models have been developed for automatic CXR interpretation. However, these models often struggle to adapt to new diagnostic tasks and complex reasoning scenarios. Recently, LLM-based agent models have emerged as a promising paradigm for CXR analysis, enhancing model's capability through tool coordination, multi-step reasoning, and team collaboration, etc. However, existing agents often rely on a single diagnostic pipeline and lack mechanisms for assessing tools' reliability, limiting their adaptability and credibility. To this end, we propose CXRAgent, a director-orchestrated, multi-stage agent for CXR interpretation, where a central director coordinates the following stages: (1) Tool Invocation: The agent strategically orchestrates a set of CXR-analysis tools, with outputs normalized and verified by the Evidence-driven Validator (EDV), which grounds diagnostic outputs with visual evidence to support reliable downstream diagnosis; (2) Diagnostic Planning: Guided by task requirements and intermediate findings, the agent formulates a targeted diagnostic plan. It then assembles an expert team accordingly, defining member roles and coordinating their interactions to enable adaptive and collaborative reasoning; (3) Collaborative Decision-making: The agent integrates insights from the expert team with accumulated contextual memories, synthesizing them into an evidence-backed diagnostic conclusion. Experiments on various CXR interpretation tasks show that CXRAgent delivers strong performance, providing visual evidence and generalizes well to clinical tasks of different complexity. Code and data are valuable at this \href{https://github.com/laojiahuo2003/CXRAgent/}{link}.

Figures (4)

• Figure 1: Comparison between prior agent models and our CXRAgent. (a) Prior agents often struggle to deal with tool conflicts in complex cases and rely on a single diagnosis pipeline. (b) CXRAgent adopts a multi-stage pipeline, guided by a core director, to flexibly coordinate tool invocation, tool-output validation, diagnostic planning, and team-integrated collaborative decision-making. Each tool's confidence is validated with visual evidence to ensure reliability.
• Figure 2: Overview of the CXRAgent. The central director coordinates the following key stages: (1) ingesting user queries, (2) orchestrating CXR-analysis tools with outputs normalized and validated by the Evidence-driven Validator (EDV) to ensure visually grounded reliability, (3) making diagnostic plans to adaptively assemble specialized expert teams for collaborative reasoning, and (4) synthesizing evidence-backed conclusions by integrating expert collaborative insights and the contextual memories.
• Figure 3: Case study of CXRAgent versus state-of-the-art methods on report generation. Top: user query, reference report, and baseline predictions. Bottom: Prediction of CXRAgent. EDV sections are in purple with the conclusion credibility hightlighted. evidence are in blue, the role and task of team members are in orange. Correct findings are in green and incorrect ones in red.
• Figure 4: Case study of CXRAgent versus state-of-the-art methods on visual question answer. Top: question, and baseline predictions. Bottom: Prediction of CXRAgent. EDV sections are in purple with the conclusion credibility highlighted. Evidence are in blue, the role and task of team members are in orange. Correct results are in green and incorrect ones in red.