Improving Zero-shot ADL Recognition with Large Language Models through Event-based Context and Confidence

TL;DR

This work addresses zero-shot ADL recognition in smart homes by replacing traditional time-based segmentation with event-based context for large language models (LLMs). By using fixed-size windows of events and repeating inferences, the approach (E-ADLLM) yields a robust activity prediction and an approximate confidence measure, enabling reliable decision-making without labeled sensor data. Evaluations on CASAS Aruba and Milan show that event-based context outperforms time-based zero-shot methods and approaches supervised event-based baselines, with confidence estimates correlating to correctness. The findings support practical deployment using open-weight models on local hardware, offering privacy advantages and promising directions for active learning and explainability in real-world health and safety applications.

Abstract

Unobtrusive sensor-based recognition of Activities of Daily Living (ADLs) in smart homes by processing data collected from IoT sensing devices supports applications such as healthcare, safety, and energy management. Recent zero-shot methods based on Large Language Models (LLMs) have the advantage of removing the reliance on labeled ADL sensor data. However, existing approaches rely on time-based segmentation, which is poorly aligned with the contextual reasoning capabilities of LLMs. Moreover, existing approaches lack methods for estimating prediction confidence. This paper proposes to improve zero-shot ADL recognition with event-based segmentation and a novel method for estimating prediction confidence. Our experimental evaluation shows that event-based segmentation consistently outperforms time-based LLM approaches on complex, realistic datasets and surpasses supervised data-driven methods, even with relatively small LLMs (e.g., Gemma 3 27B). The proposed confidence measure effectively distinguishes correct from incorrect predictions.

Figures (6)

• Figure 1: E-ADLLM overall architecture
• Figure 2: E-ADLLM's system prompt
• Figure 3: An example of a user prompt, represented in a tabular form for better readability.
• Figure 4: Output example by considering the user prompt in Figure \ref{['fig:user_prompt']} as input.
• Figure 5: Distribution of event-based windows timespan by varying the number of events $k$.