Maintenance Required: Updating and Extending Bootstrapped Human Activity Recognition Systems for Smart Homes

TL;DR

This work builds on bootstrapped HAR systems and introduces an effective updating and extension procedure for continuous improvement of HAR systems with the aim of keeping up with ever changing life circumstances.

Abstract

Developing human activity recognition (HAR) systems for smart homes is not straightforward due to varied layouts of the homes and their personalized settings, as well as idiosyncratic behaviors of residents. As such, off-the-shelf HAR systems are effective in limited capacity for an individual home, and HAR systems often need to be derived "from scratch", which comes with substantial efforts and often is burdensome to the resident. Previous work has successfully targeted the initial phase. At the end of this initial phase, we identify seed points. We build on bootstrapped HAR systems and introduce an effective updating and extension procedure for continuous improvement of HAR systems with the aim of keeping up with ever changing life circumstances. Our method makes use of the seed points identified at the end of the initial bootstrapping phase. A contrastive learning framework is trained using these seed points and labels obtained for the same. This model is then used to improve the segmentation accuracy of the identified prominent activities. Improvements in the activity recognition system through this procedure help model the majority of the routine activities in the smart home. We demonstrate the effectiveness of our procedure through experiments on the CASAS datasets that show the practical value of our approach.

Figures (5)

• Figure 1: Updating activity models for HAR system. Activity predictions from the initial bootstrapped procedure (top-right portion) are used as starting points hiremath2022bootstrapping. A self-supervision based module--SimCLR--utilized to learn representations is trained using unlabelled data, with sparse annotations from the active learning like procedure. This module is then used to provide predictions in the non-detection regions produced through the motif models in the initial bootstrapping procedure. Updated motif models are also learnt in a data incremental procedure. These modules make up the update and extension procedure (bottom portion) Predictions from both these modules lead to improved segmentation accuracy for prominent activities that form majority of routine activities in the home.
• Figure 2: Summary of the initial bootstrapping procedure for Phase 1 of the HAR system lifespan (taken with permission from hiremath2022bootstrapping)
• Figure 3: Architecture for the self-supervision module. The pre-tranining module (top) requires only unlabeled data for training. The fine-tuning module (bottom) makes use of representations from the self-supervision module and annotations to train a model for an activity recognition task. See text in Sec. \ref{['sec:sec3-self-supervision']} for details of the architecture.
• Figure 4: Data incremental procedure for update and extension of the bootstrapped model.
• Figure 5: Floor plans for Smart Homes used for our experimental evaluation: (a) CASAS-Aruba, (b) CASAS-Milan and (c) CASAS-Cairo (with permission from cook2012casas). Annotations for locations are used with permission from hiremath2022bootstrapping.