Validation of Human Pose Estimation and Human Mesh Recovery for Extracting Clinically Relevant Motion Data from Videos

TL;DR

This work evaluates marker-less human pose estimation and mesh recovery against gold-standard IMU and optical MoCap for extracting clinically relevant knee motion data from videos. Using a two-clothing-condition design and defined sit-to-stand/squat actions, it demonstrates that Mediapipe-based pose estimates can achieve close agreement with IMU/MoCap for key 1D biomarkers (minimum/maximum knee angles and ROM), while VIBE-based mesh reconstructions are more variable. The study employs robust population-level analyses (Bland–Altman, MAE, MSE, correlations) and considers practical factors like clothing and camera angles. The findings support the deployment of marker-less methods in fast-paced clinical settings and remote monitoring, while acknowledging that gold-standard devices remain superior for detailed biomechanical insights.

Abstract

This work aims to discuss the current landscape of kinematic analysis tools, ranging from the state-of-the-art in sports biomechanics such as inertial measurement units (IMUs) and retroreflective marker-based optical motion capture (MoCap) to more novel approaches from the field of computing such as human pose estimation and human mesh recovery. Primarily, this comparative analysis aims to validate the use of marker-less MoCap techniques in a clinical setting by showing that these marker-less techniques are within a reasonable range for kinematics analysis compared to the more cumbersome and less portable state-of-the-art tools. Not only does marker-less motion capture using human pose estimation produce results in-line with the results of both the IMU and MoCap kinematics but also benefits from a reduced set-up time and reduced practical knowledge and expertise to set up. Overall, while there is still room for improvement when it comes to the quality of the data produced, we believe that this compromise is within the room of error that these low-speed actions that are used in small clinical tests.

Figures (12)

• Figure 1: Diagram showing the placement of the retroreflective MoCap markers and IMU sensors in the sagittal, coronal and posterior views; allowing for lower limb kinematics, adapted from Reznick et al. 2021 reznick2021lower.
• Figure 2: Resampled and shifted plot of the left and right knee flexion curves comparing the MoCap, IMU, and Mediapipe-based joint angles for the "MoCap friendly”clothing in a sit-to-stand action
• Figure 3: Resampled and shifted plot of the left and right knee flexion curves comparing the MoCap, IMU, and Mediapipe-based joint angles for the "MoCap friendly”clothing in a squat action
• Figure 4: Resampled and shifted plot of the left and right knee flexion curves comparing the MoCap, IMU, and Mediapipe-based joint angles for the "MoCap friendly”clothing in a squat to box action
• Figure 5: Boxplots showing the range of motion for the left and right knees while wearing MoCap friendly clothing measured during the sit-to-stand, squat to box, and squat actions.