Left-Right Swapping and Upper-Lower Limb Pairing for Robust Multi-Wearable Workout Activity Detection

TL;DR

This paper tackles robust multi-wearable workout activity detection when wearable orientation can vary across and within participants. It employs a fixed, multi-window feature set with traditional CatBoost models, complemented by novel data augmentation techniques—Left-Right Swapping and Upper-Lower Limb Paring—and post-processing (k-fold majority voting, temporal smoothing, and rule-based boosting). Rotationally invariant aggregation provides insights, but the raw orientation information remains highly informative, with UL-pairing delivering the best validation macro F1 (~91.87%) and strong test-set performance. The work demonstrates that carefully designed multi-wearable augmentations and temporal processing can achieve competitive accuracy on untrimmed, real-world wearable data, offering practical guidance for robust activity recognition systems.

Abstract

This work presents the solution of the Signal Sleuths team for the 2024 HASCA WEAR challenge. The challenge focuses on detecting 18 workout activities (and the null class) using accelerometer data from 4 wearables - one worn on each limb. Data analysis revealed inconsistencies in wearable orientation within and across participants, leading to exploring novel multi-wearable data augmentation techniques. We investigate three models using a fixed feature set: (i) "raw": using all data as is, (ii) "left-right swapping": augmenting data by swapping left and right limb pairs, and (iii) "upper-lower limb paring": stacking data by using upper-lower limb pair combinations (2 wearables). Our experiments utilize traditional machine learning with multi-window feature extraction and temporal smoothing. Using 3-fold cross-validation, the raw model achieves a macro F1-score of 90.01%, whereas left-right swapping and upper-lower limb paring improve the scores to 91.30% and 91.87% respectively.

Figures (4)

• Figure 1: Median-smoothed arm accelerometer x-axis data using a 120-second smoothing window for three participants from the training set.
• Figure 2: Multi-resolution feature extraction. For each prediction timestamp, future and past window sizes of { 1s, 2s, 4s, 8s, 16s, 32s } are used, resulting in 6 $\times$ 2 feature windows. For the 1-second window, only time domain features (t) are extracted, while for all larger windows, both time and frequency domain features (t+f) are computed.
• Figure 3: Normalized confusion matrix of the grouped 3-fold predictions (with temporal prediction smoothing).
• Figure 4: Stacked prediction plot of a test participant recording. The upper three subplots show the 0.5-second predictions for each of the three training folds. The fourth subplot displays the sum of these predictions, and the resulting k-fold majority-voted prediction is shown in the fifth subplot. The blue line in the "Prediction Sum" subplot illustrates our rule-based postprocessing, identifying regions where an underrepresented class (e.g., "stretching (lunging)") is present but not selected due to another class having a higher probability (e.g., "null"). The bottom subplot displays our final prediction, incorporating these rule-based postprocessing corrections applied to the smoothed majority-voted selection from the above plot.