Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoning

Ling Yue, Sixue Xing, Jintai Chen, Tianfan Fu

TL;DR

The paper addresses the difficulty of leveraging LLMs in clinical trials due to limited access to external knowledge. It presents ClinicalAgent, a GPT-4–based, multi-agent system that uses LEAST-TO-MOST planning and ReAct reasoning to integrate DrugBank, Hetionet, and ClinicalTrials.gov data with predictive models for enrollment, safety, and efficacy. The approach achieves a PR-AUC of $0.7908$ and a ROC-AUC of $0.8347$, reflecting competitive performance and improved decision-making over standard prompting, while delivering a six-step workflow that synthesizes subproblem solutions into actionable recommendations. Despite promising results, the work acknowledges limitations related to manual agent configuration and scalability, pointing to future directions in autonomous learning and adaptive orchestration to further impact clinical trial design and execution.

Abstract

Large Language Models (LLMs) and multi-agent systems have shown impressive capabilities in natural language tasks but face challenges in clinical trial applications, primarily due to limited access to external knowledge. Recognizing the potential of advanced clinical trial tools that aggregate and predict based on the latest medical data, we propose an integrated solution to enhance their accessibility and utility. We introduce Clinical Agent System (ClinicalAgent), a clinical multi-agent system designed for clinical trial tasks, leveraging GPT-4, multi-agent architectures, LEAST-TO-MOST, and ReAct reasoning technology. This integration not only boosts LLM performance in clinical contexts but also introduces novel functionalities. The proposed method achieves competitive predictive performance in clinical trial outcome prediction (0.7908 PR-AUC), obtaining a 0.3326 improvement over the standard prompt Method. Publicly available code can be found at https://anonymous.4open.science/r/ClinicalAgent-6671.

ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoning

TL;DR

The paper addresses the difficulty of leveraging LLMs in clinical trials due to limited access to external knowledge. It presents ClinicalAgent, a GPT-4–based, multi-agent system that uses LEAST-TO-MOST planning and ReAct reasoning to integrate DrugBank, Hetionet, and ClinicalTrials.gov data with predictive models for enrollment, safety, and efficacy. The approach achieves a PR-AUC of and a ROC-AUC of , reflecting competitive performance and improved decision-making over standard prompting, while delivering a six-step workflow that synthesizes subproblem solutions into actionable recommendations. Despite promising results, the work acknowledges limitations related to manual agent configuration and scalability, pointing to future directions in autonomous learning and adaptive orchestration to further impact clinical trial design and execution.

Abstract

Large Language Models (LLMs) and multi-agent systems have shown impressive capabilities in natural language tasks but face challenges in clinical trial applications, primarily due to limited access to external knowledge. Recognizing the potential of advanced clinical trial tools that aggregate and predict based on the latest medical data, we propose an integrated solution to enhance their accessibility and utility. We introduce Clinical Agent System (ClinicalAgent), a clinical multi-agent system designed for clinical trial tasks, leveraging GPT-4, multi-agent architectures, LEAST-TO-MOST, and ReAct reasoning technology. This integration not only boosts LLM performance in clinical contexts but also introduces novel functionalities. The proposed method achieves competitive predictive performance in clinical trial outcome prediction (0.7908 PR-AUC), obtaining a 0.3326 improvement over the standard prompt Method. Publicly available code can be found at https://anonymous.4open.science/r/ClinicalAgent-6671.
Paper Structure (40 sections, 1 figure, 3 tables)

This paper contains 40 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI for Clinical Trial.
  4. Methods
  5. Overview of ClinicalAgent
  6. Agent Roles and Responsibilities
  7. Planning Agent
  8. Efficacy Agent
  9. Safety Agent
  10. Enrollment Agent
  11. Calling External Tools
  12. Data Sources
  13. Predictive AI Models
  14. Integration of Reasoning Technology
  15. Workflow
  16. ...and 25 more sections

Figures (1)

  • Figure 1: ClinicalAgent framework. Given a complex problem to solve (e.g., predicting clinical trial outcome), the role of the Planning Agent is to decompose it into three subproblems: trial enrollment, drug safety to the human body, and drug efficacy to disease. These subproblems are solved by Enrollment Agent, Safety Agent, and Efficacy Agent, respectively, enhanced by calling external tools (Section \ref{['sec:external_tool']}). Finally, the Reasoning Agent aggregates the solutions of subproblems, draws the conclusion, and makes the prediction.