Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AutoHealth: An Uncertainty-Aware Multi-Agent System for Autonomous Health Data Modeling

Tong Xia, Weibin Li, Gang Liu, Yong Li

TL;DR

AutoHealth addresses the challenge of modeling heterogeneous health data by embedding uncertainty quantification within a closed-loop, multi-agent framework. It deploys five specialized agents coordinated by a Meta-Agent to perform data exploration, task-conditioned model construction, training, and reliability-focused reporting. On a real-world benchmark of 17 health prediction tasks across six modalities, AutoHealth achieves state-of-the-art performance and substantial improvements in uncertainty estimation, demonstrating potential for safer, more interpretable deployment in healthcare. The approach emphasizes reliability, interpretability, and efficient experimentation through structured logging and cross-round memory.

Abstract

LLM-based agents have demonstrated strong potential for autonomous machine learning, yet their applicability to health data remains limited. Existing systems often struggle to generalize across heterogeneous health data modalities, rely heavily on predefined solution templates with insufficient adaptation to task-specific objectives, and largely overlook uncertainty estimation, which is essential for reliable decision-making in healthcare. To address these challenges, we propose \textit{AutoHealth}, a novel uncertainty-aware multi-agent system that autonomously models health data and assesses model reliability. \textit{AutoHealth} employs closed-loop coordination among five specialized agents to perform data exploration, task-conditioned model construction, training, and optimization, while jointly prioritizing predictive performance and uncertainty quantification. Beyond producing ready-to-use models, the system generates comprehensive reports to support trustworthy interpretation and risk-aware decision-making. To rigorously evaluate its effectiveness, we curate a challenging real-world benchmark comprising 17 tasks across diverse data modalities and learning settings. \textit{AutoHealth} completes all tasks and outperforms state-of-the-art baselines by 29.2\% in prediction performance and 50.2\% in uncertainty estimation.

AutoHealth: An Uncertainty-Aware Multi-Agent System for Autonomous Health Data Modeling

TL;DR

AutoHealth addresses the challenge of modeling heterogeneous health data by embedding uncertainty quantification within a closed-loop, multi-agent framework. It deploys five specialized agents coordinated by a Meta-Agent to perform data exploration, task-conditioned model construction, training, and reliability-focused reporting. On a real-world benchmark of 17 health prediction tasks across six modalities, AutoHealth achieves state-of-the-art performance and substantial improvements in uncertainty estimation, demonstrating potential for safer, more interpretable deployment in healthcare. The approach emphasizes reliability, interpretability, and efficient experimentation through structured logging and cross-round memory.

Abstract

LLM-based agents have demonstrated strong potential for autonomous machine learning, yet their applicability to health data remains limited. Existing systems often struggle to generalize across heterogeneous health data modalities, rely heavily on predefined solution templates with insufficient adaptation to task-specific objectives, and largely overlook uncertainty estimation, which is essential for reliable decision-making in healthcare. To address these challenges, we propose \textit{AutoHealth}, a novel uncertainty-aware multi-agent system that autonomously models health data and assesses model reliability. \textit{AutoHealth} employs closed-loop coordination among five specialized agents to perform data exploration, task-conditioned model construction, training, and optimization, while jointly prioritizing predictive performance and uncertainty quantification. Beyond producing ready-to-use models, the system generates comprehensive reports to support trustworthy interpretation and risk-aware decision-making. To rigorously evaluate its effectiveness, we curate a challenging real-world benchmark comprising 17 tasks across diverse data modalities and learning settings. \textit{AutoHealth} completes all tasks and outperforms state-of-the-art baselines by 29.2\% in prediction performance and 50.2\% in uncertainty estimation.
Paper Structure (24 sections, 5 figures, 17 tables)

This paper contains 24 sections, 5 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI for Health
  4. Reliability-aware Machine Learning
  5. LLM-based Data Science Agents
  6. AutoHealth
  7. Agent Specifications
  8. Overall Iterative Workflow
  9. Preparation Stage
  10. Planning Stage
  11. Execution Stage
  12. Reporting Stage
  13. Results
  14. Setup
  15. Main Results
  16. ...and 9 more sections

Figures (5)

  • Figure 1: AutoHealth overview. A multi-agent system that autonomously constructs predictive models and reliability-aware reports from raw health data.
  • Figure 2: AutoHealth framework. The system adopts a closed-loop, multi-agent architecture coordinated by a Meta-Agent, which orchestrates an iterative workflow spanning preparation, planning, execution, and reporting. Specialized agents operate at each stage to enable modular, reliable, and reproducible health data modeling.
  • Figure 3: Performance comparison using the SR, NPS, and CS. Avg presents the averaged score across all 17 tasks.
  • Figure 4: Excerpt of the system-generated report for Task 15 respiratory disease prediction. The complete report is provided in Appendix \ref{['sec:report']}.
  • Figure 5: Execution time proportion of each stage.