Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LifeAgentBench: A Multi-dimensional Benchmark and Agent for Personal Health Assistants in Digital Health

Ye Tian, Zihao Wang, Onat Gungor, Xiaoran Fan, Tajana Rosing

TL;DR

LifeAgentBench introduces a large-scale, cross-domain, long-horizon health reasoning QA benchmark built on lifelog data, enabling systematic evaluation of LLM-based health assistants. It reveals that current LLMs struggle with multi-domain aggregation and long-horizon reasoning, even with tool-augmented prompting. To address this, the authors propose LifeAgent, a training-free agent that decomposes complex queries into multi-step, evidence-grounded retrieval with deterministic aggregation, achieving substantial improvements over standard baselines. The work demonstrates practical potential for digital health assistants and provides a reproducible benchmark and agent framework to drive future research.

Abstract

Personalized digital health support requires long-horizon, cross-dimensional reasoning over heterogeneous lifestyle signals, and recent advances in mobile sensing and large language models (LLMs) make such support increasingly feasible. However, the capabilities of current LLMs in this setting remain unclear due to the lack of systematic benchmarks. In this paper, we introduce LifeAgentBench, a large-scale QA benchmark for long-horizon, cross-dimensional, and multi-user lifestyle health reasoning, containing 22,573 questions spanning from basic retrieval to complex reasoning. We release an extensible benchmark construction pipeline and a standardized evaluation protocol to enable reliable and scalable assessment of LLM-based health assistants. We then systematically evaluate 11 leading LLMs on LifeAgentBench and identify key bottlenecks in long-horizon aggregation and cross-dimensional reasoning. Motivated by these findings, we propose LifeAgent as a strong baseline agent for health assistant that integrates multi-step evidence retrieval with deterministic aggregation, achieving significant improvements compared with two widely used baselines. Case studies further demonstrate its potential in realistic daily-life scenarios. The benchmark is publicly available at https://anonymous.4open.science/r/LifeAgentBench-CE7B.

LifeAgentBench: A Multi-dimensional Benchmark and Agent for Personal Health Assistants in Digital Health

TL;DR

LifeAgentBench introduces a large-scale, cross-domain, long-horizon health reasoning QA benchmark built on lifelog data, enabling systematic evaluation of LLM-based health assistants. It reveals that current LLMs struggle with multi-domain aggregation and long-horizon reasoning, even with tool-augmented prompting. To address this, the authors propose LifeAgent, a training-free agent that decomposes complex queries into multi-step, evidence-grounded retrieval with deterministic aggregation, achieving substantial improvements over standard baselines. The work demonstrates practical potential for digital health assistants and provides a reproducible benchmark and agent framework to drive future research.

Abstract

Personalized digital health support requires long-horizon, cross-dimensional reasoning over heterogeneous lifestyle signals, and recent advances in mobile sensing and large language models (LLMs) make such support increasingly feasible. However, the capabilities of current LLMs in this setting remain unclear due to the lack of systematic benchmarks. In this paper, we introduce LifeAgentBench, a large-scale QA benchmark for long-horizon, cross-dimensional, and multi-user lifestyle health reasoning, containing 22,573 questions spanning from basic retrieval to complex reasoning. We release an extensible benchmark construction pipeline and a standardized evaluation protocol to enable reliable and scalable assessment of LLM-based health assistants. We then systematically evaluate 11 leading LLMs on LifeAgentBench and identify key bottlenecks in long-horizon aggregation and cross-dimensional reasoning. Motivated by these findings, we propose LifeAgent as a strong baseline agent for health assistant that integrates multi-step evidence retrieval with deterministic aggregation, achieving significant improvements compared with two widely used baselines. Case studies further demonstrate its potential in realistic daily-life scenarios. The benchmark is publicly available at https://anonymous.4open.science/r/LifeAgentBench-CE7B.
Paper Structure (17 sections, 5 figures, 4 tables)

This paper contains 17 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset
  4. Dataset Overview
  5. QA Task
  6. Design Principles
  7. Dataset Generation Pipeline
  8. LifeAgent
  9. Agent Framework
  10. Multi-step Reasoning and Decomposition
  11. Tool Design
  12. Experiments
  13. Experimental Setup
  14. Benchmarking Results
  15. Effectiveness of LifeAgent
  16. ...and 2 more sections

Figures (5)

  • Figure 1: LifeAgentBench, a QA benchmark for cross-domain, long-horizon lifestyle health reasoning, together with LifeAgent, a health assistant baseline.
  • Figure 2: Overview statistics of LifeAgentBench: (a) domain distribution for single-user and multi-user questions, (b) question word-frequency visualization, and (c) distributions of question types and answer formats.
  • Figure 3: The framework of LifeAgent.
  • Figure 4: Accuracy (%) comparison of all LLMs under two evaluation settings: CP vs. DP.
  • Figure 5: GPT-4o performance across question types (left) and answer formats (right) in CP and DP.