Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MuRAL: A Multi-Resident Ambient Sensor Dataset Annotated with Natural Language for Activities of Daily Living

Xi Chen, Julien Cumin, Fano Ramparany, Dominique Vaufreydaz

TL;DR

MuRAL introduces a context-rich, natural language–annotated dataset for multi-resident ambient sensor HAR, designed to unlock LLM reasoning in realistic smart-home settings. Collected in the DOMUS testbed, MuRAL provides long-form session narratives, per-event NL descriptions, and 27 activity labels across 21 sessions with 1–4 residents, backed by 23 ambient sensors. A two-stage LLM evaluation against LAHAR reveals that while LLMs generate semantically rich interpretations, challenges remain in disambiguating multiple residents and reasoning over sparse sensor coverage. The dataset offers a valuable benchmark for future language-driven, explainable HAR in privacy-preserving smart environments and motivates multimodal and conversational extensions.

Abstract

Recent advances in Large Language Models (LLMs) have shown promising potential for human activity recognition (HAR) using ambient sensors, especially through natural language reasoning and zero-shot learning. However, existing datasets such as CASAS, ARAS, and MARBLE were not originally designed with LLMs in mind and therefore lack the contextual richness, complexity, and annotation granularity required to fully exploit LLM capabilities. In this paper, we introduce MuRAL, the first Multi-Resident Ambient sensor dataset with natural Language, comprising over 21 hours of multi-user sensor data collected from 21 sessions in a smart-home environment. MuRAL is annotated with fine-grained natural language descriptions, resident identities, and high-level activity labels, all situated in dynamic, realistic multi-resident settings. We benchmark MuRAL using state-of-the-art LLMs for three core tasks: subject assignment, action description, and activity classification. Our results demonstrate that while LLMs can provide rich semantic interpretations of ambient data, current models still face challenges in handling multi-user ambiguity and under-specified sensor contexts. We release MuRAL to support future research on LLM-powered, explainable, and socially aware activity understanding in smart environments. For access to the dataset, please reach out to us via the provided contact information. A direct link for dataset retrieval will be made available at this location in due course.

MuRAL: A Multi-Resident Ambient Sensor Dataset Annotated with Natural Language for Activities of Daily Living

TL;DR

MuRAL introduces a context-rich, natural language–annotated dataset for multi-resident ambient sensor HAR, designed to unlock LLM reasoning in realistic smart-home settings. Collected in the DOMUS testbed, MuRAL provides long-form session narratives, per-event NL descriptions, and 27 activity labels across 21 sessions with 1–4 residents, backed by 23 ambient sensors. A two-stage LLM evaluation against LAHAR reveals that while LLMs generate semantically rich interpretations, challenges remain in disambiguating multiple residents and reasoning over sparse sensor coverage. The dataset offers a valuable benchmark for future language-driven, explainable HAR in privacy-preserving smart environments and motivates multimodal and conversational extensions.

Abstract

Recent advances in Large Language Models (LLMs) have shown promising potential for human activity recognition (HAR) using ambient sensors, especially through natural language reasoning and zero-shot learning. However, existing datasets such as CASAS, ARAS, and MARBLE were not originally designed with LLMs in mind and therefore lack the contextual richness, complexity, and annotation granularity required to fully exploit LLM capabilities. In this paper, we introduce MuRAL, the first Multi-Resident Ambient sensor dataset with natural Language, comprising over 21 hours of multi-user sensor data collected from 21 sessions in a smart-home environment. MuRAL is annotated with fine-grained natural language descriptions, resident identities, and high-level activity labels, all situated in dynamic, realistic multi-resident settings. We benchmark MuRAL using state-of-the-art LLMs for three core tasks: subject assignment, action description, and activity classification. Our results demonstrate that while LLMs can provide rich semantic interpretations of ambient data, current models still face challenges in handling multi-user ambiguity and under-specified sensor contexts. We release MuRAL to support future research on LLM-powered, explainable, and socially aware activity understanding in smart environments. For access to the dataset, please reach out to us via the provided contact information. A direct link for dataset retrieval will be made available at this location in due course.
Paper Structure (27 sections, 2 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 27 sections, 2 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. MuRAL Dataset
  4. Methodology
  5. Spatial Separability
  6. Temporal Separability
  7. Semantic Maximization
  8. Realistic Distribution
  9. Environmental Setup
  10. Data Collection
  11. Data Annotation
  12. Subject Identification and Action Description
  13. Activity Labeling
  14. Statistical Analysis
  15. Natural Language Annotation Density.
  16. ...and 12 more sections

Figures (4)

  • Figure 1: DOMUS Intelligent Apartment testbed.
  • Figure 2: Floor plan of the DOMUS smart apartment testbed with the names and locations of all ambient sensors.
  • Figure 3: Activity timeline for multiple residents in Session 01 of MuRAL.
  • Figure 4: Confusion matrices for activity classification using the baseline approach in MuRAL.