A Dataset for Multi-intensity Continuous Human Activity Recognition through Passive Sensing

TL;DR

This work presents mmDoppler, a public dataset for continuous, passive human activity recognition using a COTS mmWave radar that captures both macro and micro movements via range-doppler heatmaps and point clouds. By employing dual radar configurations and an adaptive Doppler resolution strategy, it enables robust recognition across activities from walking and running to typing and hand gestures, including multi-subject scenarios in controlled indoor environments. The dataset comprises seven subjects, nineteen activities, and over seven hours of data, with careful ground-truth labeling through video. Benchmark results using a 2D-CNN on heatmaps show high accuracy for macro (98%) and micro (95%) activities, outperforming point-cloud-based baselines for fine-grained movements, and demonstrating the value of range-doppler representations for mmWave HAR and passive sensing applications.

Abstract

Human activity recognition (HAR) is essential in healthcare, elder care, security, and human-computer interaction. The use of precise sensor data to identify activities passively and continuously makes HAR accessible and ubiquitous. Specifically, millimeter wave (mmWave) radar is promising for passive and continuous HAR due to its ability to penetrate non-metallic materials and provide high-resolution wireless sensing. Although mmWave sensors are effective at capturing macro-scale activities, like exercising, they fail to capture micro-scale activities, such as typing. In this paper, we introduce mmDoppler, a novel dataset that utilizes off-the-shelf (COTS) mmWave radar in order to capture both macro and micro-scale human movements using a machine-learning driven signal processing pipeline. The dataset includes seven subjects performing 19 distinct activities and employs adaptive doppler resolution to enhance activity recognition. By adjusting the radar's doppler resolution based on the activity type, our system captures subtle movements more precisely. mmDoppler includes range-doppler heatmaps, offering detailed motion dynamics, with data collected in a controlled environment with single as well as multiple subjects performing activities simultaneously. The dataset aims to bridge the gap in HAR systems by providing a more comprehensive and detailed resource for improving the robustness and accuracy of mmWave radar activity recognition.

Figures (9)

• Figure 1: IWR1642 hardware based setup and data collection in different rooms (R1, R2, R3).
• Figure 2: Dataset Distribution per activities (in $\%age$) (a) Macro Activities, (b) Micro Activities.
• Figure 3: Standard deviation (std) in the range-doppler heatmaps captured during the entire activity duration. (a)-(j): Macro activities with low doppler resolution, (k)-(s): Micro activities with high doppler resolution. Activities having similar body movements have similar patterns, but the difference can be captured in the temporal domain.
• Figure 4: Range-doppler signatures (standard deviation for individual activity windows) over time.
• Figure 5: Range Doppler signature of two subjects performing different activities simultaneously