Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Tasks

Hengjian Gao, Kaiwei Zhang, Shibo Wang, Mingjie Chen, Qihang Cao, Xianfeng Wang, Yucheng Zhu, Xiongkuo Min, Wei Sun, Dandan Zhu, Guangtao Zhai

TL;DR

LifeEval is introduced, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective, highlighting essential directions for advancing human-centered interactive intelligence.

Abstract

The rapid progress of Multimodal Large Language Models (MLLMs) marks a significant step toward artificial general intelligence, offering great potential for augmenting human capabilities. However, their ability to provide effective assistance in dynamic, real-world environments remains largely underexplored. Existing video benchmarks predominantly assess passive understanding through retrospective analysis or isolated perception tasks, failing to capture the interactive and adaptive nature of real-time user assistance. To bridge this gap, we introduce LifeEval, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective. LifeEval emphasizes three key aspects: task-oriented holistic evaluation, egocentric real-time perception from continuous first-person streams, and human-assistant collaborative interaction through natural dialogues. Constructed via a rigorous annotation pipeline, the benchmark comprises 4,075 high-quality question-answer pairs across 6 core capability dimensions. Extensive evaluations of 26 state-of-the-art MLLMs on LifeEval reveal substantial challenges in achieving timely, effective and adaptive interaction, highlighting essential directions for advancing human-centered interactive intelligence.

LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Tasks

TL;DR

LifeEval is introduced, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective, highlighting essential directions for advancing human-centered interactive intelligence.

Abstract

The rapid progress of Multimodal Large Language Models (MLLMs) marks a significant step toward artificial general intelligence, offering great potential for augmenting human capabilities. However, their ability to provide effective assistance in dynamic, real-world environments remains largely underexplored. Existing video benchmarks predominantly assess passive understanding through retrospective analysis or isolated perception tasks, failing to capture the interactive and adaptive nature of real-time user assistance. To bridge this gap, we introduce LifeEval, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective. LifeEval emphasizes three key aspects: task-oriented holistic evaluation, egocentric real-time perception from continuous first-person streams, and human-assistant collaborative interaction through natural dialogues. Constructed via a rigorous annotation pipeline, the benchmark comprises 4,075 high-quality question-answer pairs across 6 core capability dimensions. Extensive evaluations of 26 state-of-the-art MLLMs on LifeEval reveal substantial challenges in achieving timely, effective and adaptive interaction, highlighting essential directions for advancing human-centered interactive intelligence.
Paper Structure (23 sections, 5 figures, 4 tables)

This paper contains 23 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. General Video Understanding Benchmarks
  4. Egocentric Video Understanding Benchmarks
  5. LifeEval
  6. Capability Taxonomy
  7. Benchmark Construction
  8. Video Collection
  9. Data Curation Pipeline
  10. Evaluation Metrics
  11. Benchmark Statistics
  12. Experiments
  13. Experimental Setup
  14. Main Results
  15. Proprietary MLLMs.
  16. ...and 8 more sections

Figures (5)

  • Figure 1: Overview of LifeEval. The benchmark emphasizes real-life interactive scenarios, requiring models to deliver precise, context-aware, and adaptive assistance.
  • Figure 2: Capability dimensions and examples in LifeEval. LifeEval defines six core capability dimensions to comprehensively evaluate a model’s interactive assistance abilities in daily life. The benchmark features both multiple-choice and open-ended questions, each accompanied by detailed reasoning explanations.
  • Figure 3: Task goal distribution and QA vocabulary statistics in LifeEval.
  • Figure 4: Performance comparison of representative MLLMs on LifeEval. Left: Evaluation results across six capability dimensions. Right: Evaluation results across five everyday scenarios.
  • Figure 5: Relationship between model parameter scale and performance on LifeEval for open-source MLLM families.