Generative Pretrained Embedding and Hierarchical Irregular Time Series Representation for Daily Living Activity Recognition

TL;DR

This work tackles daily living activity recognition (HAR) in smart homes using ambient sensors with irregular sampling and long-range dependencies. It contrasts a Transformer decoder-based sensor embedding (GPT-like) against ELMo embeddings and couples them with a hierarchical, multi-timescale architecture augmented by an hour-of-day time encoding to capture intra- and inter-activity dynamics. Through experiments on three CASAS datasets (Aruba, Milan, Cairo), the GPT-based embeddings within the GPTHAR framework achieve superior classification performance, with time encoding further enhancing accuracy and stability. The findings demonstrate that a GPT-inspired hierarchical representation effectively models cause-and-effect relations between activities, offering a path toward more accurate and temporally aware ambient HAR in real-world smart homes.

Abstract

Within the evolving landscape of smart homes, the precise recognition of daily living activities using ambient sensor data stands paramount. This paper not only aims to bolster existing algorithms by evaluating two distinct pretrained embeddings suited for ambient sensor activations but also introduces a novel hierarchical architecture. We delve into an architecture anchored on Transformer Decoder-based pre-trained embeddings, reminiscent of the GPT design, and contrast it with the previously established state-of-the-art (SOTA) ELMo embeddings for ambient sensors. Our proposed hierarchical structure leverages the strengths of each pre-trained embedding, enabling the discernment of activity dependencies and sequence order, thereby enhancing classification precision. To further refine recognition, we incorporate into our proposed architecture an hour-of-the-day embedding. Empirical evaluations underscore the preeminence of the Transformer Decoder embedding in classification endeavors. Additionally, our innovative hierarchical design significantly bolsters the efficacy of both pre-trained embeddings, notably in capturing inter-activity nuances. The integration of temporal aspects subtly but distinctively augments classification, especially for time-sensitive activities. In conclusion, our GPT-inspired hierarchical approach, infused with temporal insights, outshines the SOTA ELMo benchmark.

Figures (14)

• Figure 1: Model architecture of GPTAR and its GPT transformer decoder. GPTAR embeds the sensor signal with 3 layers of GPT transformer decoder embedding and a bi-LSTM. The illustration of the Transformer decoder was inspired by vaswani2017attention
• Figure 2: Complete architecture of the Generative Pre-trained Transformer for Hierarchical Activity Recognition (GPTHAR), composed of 3 low-level modules to compute 3 successive activities, and a top-level composed of a bi-LSTM and a softmax classifier. The low-level module processes in parallel the hour timestamp with a bi-LSTM and the sensor signal with a GPT transformer decoder embedding and a bi-LSTM
• Figure 3: Confusion matrices per algorithm on the Milan dataset
• Figure 4: Confusion matrices per algorithm on the Cairo dataset
• Figure 5: Visualization of Activity Embeddings for the Cairo Dataset Generated by ELMo and GPT