Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EMBC Special Issue: Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture

Seth Donahue, Irina Djuraskovic, Kunal Shah, Fabian Sinz, Ross Chafetz, R. James Cotton

TL;DR

The paper addresses the need for calibrated uncertainty in video-based multiview markerless motion capture (MMMC) for clinical gait analysis. It extends prior probabilistic MMMC by externally validating calibrated outputs against clinical standards across two sites, using an instrumented walkway for spatiotemporal metrics and marker-based kinematics for joint angles. Through variational inference, it estimates per-time joint-angle posteriors $q_ ho(\boldsymbol \theta_t)$ with an ELBO-based objective and $ECE$-driven calibration, quantified via $PIT$ analyses. Results show well-calibrated spatial metrics ($$ECE \,=\ 0.05\) and generally well-calibrated kinematics after bias correction, with uncertainty estimates effectively signaling unreliability and enabling automatic filtering of low-quality data for clinically deployable gait analysis.

Abstract

Video-based human movement analysis holds potential for movement assessment in clinical practice and research. However, the clinical implementation and trust of multi-view markerless motion capture (MMMC) require that, in addition to being accurate, these systems produce reliable confidence intervals to indicate how accurate they are for any individual. Building on our prior work utilizing variational inference to estimate joint angle posterior distributions, this study evaluates the calibration and reliability of a probabilistic MMMC method. We analyzed data from 68 participants across two institutions, validating the model against an instrumented walkway and standard marker-based motion capture. We measured the calibration of the confidence intervals using the Expected Calibration Error (ECE). The model demonstrated reliable calibration, yielding ECE values generally < 0.1 for both step and stride length and bias-corrected gait kinematics. We observed a median step and stride length error of ~16 mm and ~12 mm respectively, with median bias-corrected kinematic errors ranging from 1.5 to 3.8 degrees across lower extremity joints. Consistent with the calibrated ECE, the magnitude of the model's predicted uncertainty correlated strongly with observed error measures. These findings indicate that, as designed, the probabilistic model reconstruction quantifies epistemic uncertainty, allowing it to identify unreliable outputs without the need for concurrent ground-truth instrumentation.

EMBC Special Issue: Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture

TL;DR

The paper addresses the need for calibrated uncertainty in video-based multiview markerless motion capture (MMMC) for clinical gait analysis. It extends prior probabilistic MMMC by externally validating calibrated outputs against clinical standards across two sites, using an instrumented walkway for spatiotemporal metrics and marker-based kinematics for joint angles. Through variational inference, it estimates per-time joint-angle posteriors with an ELBO-based objective and -driven calibration, quantified via analyses. Results show well-calibrated spatial metrics ($$ECE \,=\ 0.05\) and generally well-calibrated kinematics after bias correction, with uncertainty estimates effectively signaling unreliability and enabling automatic filtering of low-quality data for clinically deployable gait analysis.

Abstract

Video-based human movement analysis holds potential for movement assessment in clinical practice and research. However, the clinical implementation and trust of multi-view markerless motion capture (MMMC) require that, in addition to being accurate, these systems produce reliable confidence intervals to indicate how accurate they are for any individual. Building on our prior work utilizing variational inference to estimate joint angle posterior distributions, this study evaluates the calibration and reliability of a probabilistic MMMC method. We analyzed data from 68 participants across two institutions, validating the model against an instrumented walkway and standard marker-based motion capture. We measured the calibration of the confidence intervals using the Expected Calibration Error (ECE). The model demonstrated reliable calibration, yielding ECE values generally < 0.1 for both step and stride length and bias-corrected gait kinematics. We observed a median step and stride length error of ~16 mm and ~12 mm respectively, with median bias-corrected kinematic errors ranging from 1.5 to 3.8 degrees across lower extremity joints. Consistent with the calibrated ECE, the magnitude of the model's predicted uncertainty correlated strongly with observed error measures. These findings indicate that, as designed, the probabilistic model reconstruction quantifies epistemic uncertainty, allowing it to identify unreliable outputs without the need for concurrent ground-truth instrumentation.
Paper Structure (22 sections, 3 equations, 5 figures, 5 tables)

This paper contains 22 sections, 3 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Computer Vision and Differentiable Models
  3. Probabilistic Modeling and Calibration
  4. Purpose and key contributions
  5. Methods
  6. Shirley Ryan Participants
  7. Shriners Participants
  8. Marker-based Data Processing
  9. Markerless Data Collection and Preprocessing
  10. Markerless Probabilistic Calculations and Hyperparameter Optimization
  11. Expected Calibration Error
  12. Spatial Errors and GaitRite Validation
  13. Joint Angle Errors and Validation
  14. Results
  15. Spatiotemporal Gait Metrics Calibration and Accuracy
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Overview of the Probabilistic MMMC Validation Pipeline. The pipeline proceeds in four stages: (1) Multiview Data Acquisition collects synchronized video from varying camera configurations alongside either marker-based or GaitRite references across diverse cohorts, including pediatric, neurologic, and prosthetic users. (2) Probabilistic Modeling estimates pose parameters, outputting joint angles and uncertainty in the measures. (3) Outputs from the probabilistic model are the expected (mean of the posterior) trajectory and associated per-time and per-joint standard deviation, which can be presented as a 95% CI. (4) Model Calibration & Validation evaluates the system's spatial accuracy against GaitRite ground truth (Step Length Distribution) and assesses the reliability of the predicted confidence intervals using Probability Integral Transform (PIT) analysis (Kinematic Calibration). (Figure generated with assistance of Google Nano Banana)
  • Figure 2: Empirical calibration curve for step length and stride length measures across all participants (blue) and separated by participant groups in different colors. PIT values (dots) are obtained from the posterior predictive absolute error distribution and are plotted against the expected uniform distribution (dashed line). A perfectly calibrated model will have the PIT values fall on the identity line. Deviations from the identity line are indications of miscalculation quantified by ECE (See Table \ref{['tab:step_length_ece_table']}).
  • Figure 3: Step length and stride length mean absolute error versus the predicted uncertainty across all participants. The points show the mean absolute error for samples that belong in that uncertainty bin. Shaded regions show the 95% confidence intervals for the mean absolute error in each uncertainty bin
  • Figure 4: Sagittal-plane joint kinematics and uncertainty calibration results for pelvis tilt, hip flexion, knee flexion, and ankle dorsiflexion. Left column: Probabilistic model's predictions are shown as the mean trajectory (solid blue) with 95% confidence intervals (shaded region). Marker-based kinematics are shown for reference (dashed gray). Right column: Empirical values from the PIT, CDF and the dashed diagonal line indicates perfect calibration.
  • Figure 5: Percentage of marker-based joint angles that fall within the probabilistic model’s nominal confidence intervals (25%, 50%, 75%, and 95%) across eight lower-extremity kinematic variables, on a per-trial basis. Each panel corresponds to a different nominal confidence level, with the dashed horizontal line indicating ideal coverage. This figure summarizes how well the model’s predicted uncertainty aligns with ground-truth marker-based kinematics across joints and confidence levels.