Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot

TL;DR

This work tackles real-time human activity recognition for healthcare monitoring using a movable mmWave radar on a robot-mounted edge platform. It introduces RobHAR, combining a sparse point cloud global embedding via Light-PointNet with a bidirectional lightweight LSTM (BiLiLSTM) to capture spatio-temporal patterns, and strengthens robustness with a novel HMM-CTC transition-optimization layer for continuous HAR. A segment-wise point cloud augmentation (SPCA) and adaptive segment alignment (AS) enhance learning from sparse radar data, while extensive experiments on MMActivity, discHAR, and contHAR demonstrate superior accuracy and efficiency, including deployment on edge hardware. The approach yields a mobile, privacy-preserving healthcare monitoring robot capable of accurate and continuous HAR in real-world environments, with favorable computational characteristics suitable for edge devices.

Abstract

Healthcare monitoring is crucial, especially for the daily care of elderly individuals living alone. It can detect dangerous occurrences, such as falls, and provide timely alerts to save lives. Non-invasive millimeter wave (mmWave) radar-based healthcare monitoring systems using advanced human activity recognition (HAR) models have recently gained significant attention. However, they encounter challenges in handling sparse point clouds, achieving real-time continuous classification, and coping with limited monitoring ranges when statically mounted. To overcome these limitations, we propose RobHAR, a movable robot-mounted mmWave radar system with lightweight deep neural networks for real-time monitoring of human activities. Specifically, we first propose a sparse point cloud-based global embedding to learn the features of point clouds using the light-PointNet (LPN) backbone. Then, we learn the temporal pattern with a bidirectional lightweight LSTM model (BiLiLSTM). In addition, we implement a transition optimization strategy, integrating the Hidden Markov Model (HMM) with Connectionist Temporal Classification (CTC) to improve the accuracy and robustness of the continuous HAR. Our experiments on three datasets indicate that our method significantly outperforms the previous studies in both discrete and continuous HAR tasks. Finally, we deploy our system on a movable robot-mounted edge computing platform, achieving flexible healthcare monitoring in real-world scenarios.

Figures (6)

• Figure 1: The architecture of our proposed RobHAR system.
• Figure 2: The sparse point cloud global embedding model to learn the frame-wise point cloud feature.
• Figure 3: The statistics of valid samples of the discHAR and contHAR dataset. The $L$ denotes the size of the time window. With different $L$ values, the valid samples in both datasets change dramatically.
• Figure 4: The impact of the time window $L$ on the accuracy and execution time on the MMActivity dataset
• Figure 5: The statistics and impact of number of points per frame on the MMActivity dataset