Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Monocular Video-Based Gait Analysis Using Motion Imitation with Physics-Based Simulation

Nikolaos Smyrnakis, Tasos Karakostas, R. James Cotton

TL;DR

Monocular gait analysis from smartphone video often yields physically implausible estimates due to unconstrained inference. The authors address this by training a reinforcement learning policy to drive a physics-based humanoid in Brax to imitate observed movements, enforcing biomechanical constraints and improving walking velocity and step length estimates. Relative to a prior model-free approach, the method achieves higher correlations and lower RMSE for key spatiotemporal parameters and joint angles, though some residual errors remain due to model simplifications. The work demonstrates the potential of physics-constrained, imitation-based inference for clinical gait assessment and outlines clear paths for richer biomechanical modeling and broader validation.

Abstract

Gait analysis from videos obtained from a smartphone would open up many clinical opportunities for detecting and quantifying gait impairments. However, existing approaches for estimating gait parameters from videos can produce physically implausible results. To overcome this, we train a policy using reinforcement learning to control a physics simulation of human movement to replicate the movement seen in video. This forces the inferred movements to be physically plausible, while improving the accuracy of the inferred step length and walking velocity.

Advancing Monocular Video-Based Gait Analysis Using Motion Imitation with Physics-Based Simulation

TL;DR

Monocular gait analysis from smartphone video often yields physically implausible estimates due to unconstrained inference. The authors address this by training a reinforcement learning policy to drive a physics-based humanoid in Brax to imitate observed movements, enforcing biomechanical constraints and improving walking velocity and step length estimates. Relative to a prior model-free approach, the method achieves higher correlations and lower RMSE for key spatiotemporal parameters and joint angles, though some residual errors remain due to model simplifications. The work demonstrates the potential of physics-constrained, imitation-based inference for clinical gait assessment and outlines clear paths for richer biomechanical modeling and broader validation.

Abstract

Gait analysis from videos obtained from a smartphone would open up many clinical opportunities for detecting and quantifying gait impairments. However, existing approaches for estimating gait parameters from videos can produce physically implausible results. To overcome this, we train a policy using reinforcement learning to control a physics simulation of human movement to replicate the movement seen in video. This forces the inferred movements to be physically plausible, while improving the accuracy of the inferred step length and walking velocity.
Paper Structure (23 sections, 5 equations, 4 figures, 1 table)

This paper contains 23 sections, 5 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Gait Laboratory Dataset
  4. Video processing
  5. Marker-based gait analysis
  6. Trial selection
  7. Brax model design
  8. Trajectory following environment
  9. Reinforcement learning and control policy
  10. Augmentation
  11. Initializer
  12. Hyperparameter Tuning
  13. Final implementation details
  14. Performance Metrics
  15. Results
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Waveforms from a single trial. The top row shows the sagittal plane position of several keypoints on the left side. The remaining rows show the pelvis velocity, hip angles, knee angles, and ankle positions. The mocap data is shown with dashed lines and solid lines are the model predictions.
  • Figure 2: Evenly spaced frames from one second of gait data from the same trial as Figure \ref{['fig:rollout']}.
  • Figure 3: Performance statistics estimating step length, step width, and velocity in the test data. Each metric is shown in a respective column. The top row show the correlations between the ground truth and predicted gait parameters. The bottom row shows the Bland-Altman plots, with the median difference and interquartile range indicated by horizontal bars.
  • Figure 4: Histogram of timing errors for foot contact events.