KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture

TL;DR

KinTwin tackles the problem of inferring internal biomechanics (torques, ground reaction forces, and muscle activations) from markerless motion capture by training imitation-learning policies on biomechanical models that include both torque-driven and 92-muscle lower-limb systems. The approach uses PPO to learn a goal-conditioned policy that tracks target trajectories with residual force control and a rich observation structure, enabling estimation of clinically relevant kinetics rather than just kinematics. Key results show accurate kinematic replication across a spectrum of able-bodied and impaired movements, with ground-truth validation against instrumented-walkway data and sensitivity to clinically meaningful differences in torque and muscle activations. This work advances digital-twin style movement analytics for clinical practice, offering a path toward high-fidelity phenotyping and personalized rehabilitation tools, while acknowledging limitations in data quality, external forces, and the need for broader validation.

Abstract

Broader access to high-quality movement analysis could greatly benefit movement science and rehabilitation, such as allowing more detailed characterization of movement impairments and responses to interventions, or even enabling early detection of new neurological conditions or fall risk. While emerging technologies are making it easier to capture kinematics with biomechanical models, or how joint angles change over time, inferring the underlying physics that give rise to these movements, including ground reaction forces, joint torques, or even muscle activations, is still challenging. Here we explore whether imitation learning applied to a biomechanical model from a large dataset of movements from able-bodied and impaired individuals can learn to compute these inverse dynamics. Although imitation learning in human pose estimation has seen great interest in recent years, our work differences in several ways: we focus on using an accurate biomechanical model instead of models adopted for computer vision, we test it on a dataset that contains participants with impaired movements, we reported detailed tracking metrics relevant for the clinical measurement of movement including joint angles and ground contact events, and finally we apply imitation learning to a muscle-driven neuromusculoskeletal model. We show that our imitation learning policy, KinTwin, can accurately replicate the kinematics of a wide range of movements, including those with assistive devices or therapist assistance, and that it can infer clinically meaningful differences in joint torques and muscle activations. Our work demonstrates the potential for using imitation learning to enable high-quality movement analysis in clinical practice.

Figures (13)

• Figure 1: Waveforms from a single trial from a participant using a left transfemoral prosthesis. Our color convention is blue for left and red for right throughout. The top row left and center columns show the target trajectory hip and knee angles respectively, in a dashed black trace, which is hardly visible due to the close tracking. The next rows show the hip and knee velocities. The last row shows the internal moments. The right column shows the ground reaction forces and detects greater forward propulsion at terminal stance from the intact compared to the prosthetic side. The black dots correspond to gait events from the instrumented walkway, and the crosses correspond to foot off events. The bottom row compares the progression of the center of pressure from the physics simulation to the line between heel and toe positions from the instrumented walkway, showing very close agreement.
• Figure 2: Histogram of errors for foot contact events, foot off events, and stride lengths, compared to an instrumented walkway.
• Figure 3: Example muscle-driven model replicating walking from someone with a right (red traces) hemiparetic gait after a stroke. The top panels for the hip and knee show the target trajectory with a dashed black line, showing that the rollout closely replicates the target kinematics with their asymmetric pattern. The bottom traces show the muscle activations. For example, it is apparent that the semimembranosus (hamstring) shows reduced activation on the right side, corresponding to the reduced knee flexion on this side. Similarly, the tibialis anterior activation is inferred when initiating the swing phase to clear the toes on and during weight acceptance to prevent foot slap while capturing the asymmetric dynamics and magnitude.
• Figure 4: Example of a failure for the imitation learning policy to replicate a pediatric walking sample using a rolling walker. Rendering from the target trajectory is shown on the left, and the policy tripping is shown on the right. Note the large posteriorly directed ground reaction forces as the foot catches the ground and initates a trip.
• Figure : Six participants with no gait impairments. While some slight asymmetries are noted, likely due to personal walking patterns, these asymmetries are much smaller than the following participants with gait impairments.