MicroXercise: A Micro-Level Comparative and Explainable System for Remote Physical Therapy

TL;DR

MicroXercise tackles the challenge of delivering high-fidelity, explainable feedback for remote shoulder physical therapy by fusing micro-motion analysis from wearable IMU data with a multi-dimensional DTW-aligned spatiotemporal Siamese network and attribution-based explanations. The system translates sensor signals into actionable feedback across video, text, and avatar modalities, enabling precise micro-motion guidance and simultaneous assessment of range of motion and stability. Key contributions include a micro-motion syncing pipeline with primitive removal, adaptive DTW, and micro-segmentation; a multi-task spatiotemporal Siamese Neural Network with attribution extraction; and template-based text generation, all validated on a shoulder PT dataset with notable interpretability gains. Evaluation shows Feature Mutual Information improvements of about 39% and Continuity improvements of about 42% over baselines, indicating enhanced explainability and fidelity with practical potential for improving home-based PT adherence and outcomes, while also motivating future work on broader exercises, user studies, and data privacy considerations.

Abstract

Recent global estimates suggest that as many as 2.41 billion individuals have health conditions that would benefit from rehabilitation services. Home-based Physical Therapy (PT) faces significant challenges in providing interactive feedback and meaningful observation for therapists and patients. To fill this gap, we present MicroXercise, which integrates micro-motion analysis with wearable sensors, providing therapists and patients with a comprehensive feedback interface, including video, text, and scores. Crucially, it employs multi-dimensional Dynamic Time Warping (DTW) and attribution-based explainable methods to analyze the existing deep learning neural networks in monitoring exercises, focusing on a high granularity of exercise. This synergistic approach is pivotal, providing output matching the input size to precisely highlight critical subtleties and movements in PT, thus transforming complex AI analysis into clear, actionable feedback. By highlighting these micro-motions in different metrics, such as stability and range of motion, MicroXercise significantly enhances the understanding and relevance of feedback for end-users. Comparative performance metrics underscore its effectiveness over traditional methods, such as a 39% and 42% improvement in Feature Mutual Information (FMI) and Continuity. MicroXercise is a step ahead in home-based physical therapy, providing a technologically advanced and intuitively helpful solution to enhance patient care and outcomes.

Figures (7)

• Figure 1: This diagram illustrates the process of exercise performance and feedback generation using our system. The user performs an exercise while wearing a smartwatch, which generates a off-site exercise (or "signal exercise") sent to the user's smartphone in (1). The smartphone accesses the off-site exercise and the on-site exercise ("anchor exercise"), which is supervised by the therapist in clinics and stored in the cloud, then processes these two exercises together and sends them to the micro-motion algorithm for analyzing in cloud (2). This provides explainable feedback, such as the range of motion and stability, and micro-motion feedback in text and visualization (3). Next, the user will have access to the generated feedback, as shown in (4). Additionally, the therapists can view users' feedback and exercise history, as shown in (5).
• Figure 2: MicroXercise System Pipeline: A schematic representation of the exercise analysis pipeline. The process begins with the users performing several repetitions of an exercise, pre-segmented into individual repetitions (light blue). The anchor exercise, considered the ground truth, is colored in orange. The data undergoes a series of processing steps, including primitive (noise) removal, adaptive DTW, and micro-segmentation. In parallel, utilizing an existing comparative neural network (such as Siamese Neural Network), MicroXercise Monitoring produces an attribution map from attribution-based methods. It aids in the video generation process using inverse kinematics and in the text generation. The final video emphasizes key features pinpointed by the attribution map with in-depth, granular feedback on their exercise metrics.
• Figure 3: An illustrative diagram to show the comparison of the explainable AI system with Adaptive DTW and segmentation on signal (top) and anchor (bottom) exercises, with video recording, in one repetition. This diagram has seven rows: 3 axial accelerometer, 3 axial gyroscope, and reference video recording. The signals are also shown in blue, and heatmaps are shown in dark blue.
• Figure 4: Visual Explainable Results: This result encapsulates critical elements such as the user's similarity score relative to the anchor exercises, temporal fluctuations in stability, and the discrepancy in range of motion at the apex of particular movements. These insights are derived from post-processed attributions generated by our micro-motion analysis.
• Figure 5: Signal comparison with raw attribution versus modified attribution in the attribution produced by IG. As shown in the figure, column 1 shows its original signal exercise. The raw attribution is very noisy and inconsistent in column 2 and 3, but the modified attribution produces more consistent results as shown in column 4 and 5.