DS-MS-TCN: Otago Exercises Recognition with a Dual-Scale Multi-Stage Temporal Convolutional Network

TL;DR

This study addresses robust recognition of Otago Exercises in daily life using a single waist-worn IMU. It introduces DS-MS-TCN, a dual-scale, multi-stage temporal convolutional network that first classifies micro-labels (repetitions) and then macro-labels (entire exercise segments), with refinement stages to reduce over-segmentation. The approach, trained with a novel micro-label annotation scheme and a combined loss function, achieves f1-scores above 80% and IoU-based f1-scores above 60% across four exercises, outperforming several baselines and avoiding post-processing. Generalization to home scenarios shows a decline due to real-world variability, but the method demonstrates strong potential for reliable OEP recognition in daily living and contributes a new perspective on repetition-level HAR. Future work includes transfer/semi-supervised learning to mitigate annotation workload and potential integration with video-based methods to further boost performance.

Abstract

The Otago Exercise Program (OEP) represents a crucial rehabilitation initiative tailored for older adults, aimed at enhancing balance and strength. Despite previous efforts utilizing wearable sensors for OEP recognition, existing studies have exhibited limitations in terms of accuracy and robustness. This study addresses these limitations by employing a single waist-mounted Inertial Measurement Unit (IMU) to recognize OEP exercises among community-dwelling older adults in their daily lives. A cohort of 36 older adults participated in laboratory settings, supplemented by an additional 7 older adults recruited for at-home assessments. The study proposes a Dual-Scale Multi-Stage Temporal Convolutional Network (DS-MS-TCN) designed for two-level sequence-to-sequence classification, incorporating them in one loss function. In the first stage, the model focuses on recognizing each repetition of the exercises (micro labels). Subsequent stages extend the recognition to encompass the complete range of exercises (macro labels). The DS-MS-TCN model surpasses existing state-of-the-art deep learning models, achieving f1-scores exceeding 80% and Intersection over Union (IoU) f1-scores surpassing 60% for all four exercises evaluated. Notably, the model outperforms the prior study utilizing the sliding window technique, eliminating the need for post-processing stages and window size tuning. To our knowledge, we are the first to present a novel perspective on enhancing Human Activity Recognition (HAR) systems through the recognition of each repetition of activities.

Figures (10)

• Figure 1: The accelerometer signals and annotation of macro labels and micro labels for chair rising, where each segment of macro activity consisted of five repetitions of sit-to-stand (green blocks) and stand-to-sit (red blocks)
• Figure 2: Examples of each repetition of micro activities and corresponding acceleration. The shadowed area illustrates the positive micro labels and the unshadowed area illustrates other labels
• Figure 3: The overview of the proposed DS-MS-TCN. Four stages of SS-TCN were applied to comply to classify micro labels and macro labels. Each SS-TCN composed of multiple residual dilated layers.
• Figure 4: The definition of IoU, TP, FP, and FN. There are eight cases shown in the figure. In case 4 and case 5, FN or FP depends on the length of the true and predicted segment. In case 7, if a true segment is predicted as some smaller segments, FP numbers increase. In case 8, if some separate true segments are predicted as one segment, FN numbers increase.
• Figure 5: F1-scores and IoU f1-scores applying different loss weight for the first stage (i.e. $\eta$)