WearableMil: An End-to-End Framework for Military Activity Recognition and Performance Monitoring

TL;DR

This work tackles the problem of continuous military activity recognition and fatigue monitoring using wearable sensors to prevent musculoskeletal injuries. It proposes an end-to-end framework combining physiologically-informed imputation, a unified data processing pipeline, and a hierarchical bidirectional LSTM with a hierarchical focal loss to classify coarse and fine-grained activities. Key contributions include a novel sleep-imputation method guided by physiological signals, a scalable multimodal data fusion approach via a Linear Truncated Model, and a real-time visualization tool for benchmarking individual performance against group norms. The approach yields strong within-subject performance (e.g., Level 1 accuracy up to 93.8%) and offers practical insights for training optimization and injury prevention in military settings, albeit with challenges in cross-user generalization and short-duration activity detection.

Abstract

Musculoskeletal injuries during military training significantly impact readiness, making prevention through activity monitoring crucial. While Human Activity Recognition (HAR) using wearable devices offers promising solutions, it faces challenges in processing continuous data streams and recognizing diverse activities without predefined sessions. This paper introduces an end-to-end framework for preprocessing, analyzing, and recognizing activities from wearable data in military training contexts. Using data from 135 soldiers wearing \textit{Garmin--55} smartwatches over six months with over 15 million minutes. We develop a hierarchical deep learning approach that achieves 93.8% accuracy in temporal splits and 83.8% in cross-user evaluation. Our framework addresses missing data through physiologically-informed methods, reducing unknown sleep states from 40.38% to 3.66%. We demonstrate that while longer time windows (45-60 minutes) improve basic state classification, they present trade-offs in detecting fine-grained activities. Additionally, we introduce an intuitive visualization system that enables real-time comparison of individual performance against group metrics across multiple physiological indicators. This approach to activity recognition and performance monitoring provides military trainers with actionable insights for optimizing training programs and preventing injuries.

Figures (4)

• Figure 1: Overview of the Data Processing and Prediction Pipeline. The pipeline integrates multi-source wearable data, including HR, physical activity, and sleep data, into a unified grid using the LTM. Missing sleep data is imputed leveraging physiological insights, HR, and physical activity thresholds. Windows of varying sizes are labeled and inputted into a neural network for sequential activity prediction.
• Figure 2: Performance Trends Across Window Sizes. The 2x2 grid compares Level 1 (L1) and Level 2 (L2) metrics for temporal (blue) and user (red) splits across varying window sizes. Temporal splits consistently outperform user splits across all metrics. L1 metrics (top row) remain stable for temporal splits, while user splits exhibit variability. For L2 metrics (bottom row), temporal splits peak at 45 minutes, particularly in F1 score and ROC AUC, before declining slightly at 60 minutes. User splits show stable but lower performance due to challenges in generalizing to unseen users. The uniform y-axis range (50--100%) ensures comparability across subplots.
• Figure 3: Confusion matrix for the 30-minute temporal split (Level 1 and 2).
• Figure 4: Individual performance across tasks (a-c), represented by the blue line, is compared to the group median (red line) across multiple metrics, including distance per minute, steps per minute, and pulse ratios. Solid lines indicate median values, while shaded areas show variability (data range) for each metric, reflecting consistency—larger shaded areas indicate higher variability.