Learning Speed-Adaptive Walking Agent Using Imitation Learning with Physics-Informed Simulation

TL;DR

This paper tackles the sim-to-real gap in digital twins of human gait by developing a speed-adaptive skeletal walking agent trained with synthetic biomechanical data and adversarial imitation learning. It combines a synthetic motion data generator derived from open-source gait datasets with Variational Adversarial Imitation Learning (VAIL) and a curriculum scheme to enable robust speed generalization. The authors perform extensive analyses comparing the agent's kinematics to ground-truth biomechanics data and identify optimal training settings that balance imitation quality with speed tracking, achieving high fidelity across speeds. The work advances digital twin applications in biomechanics, assistive device design, and rehabilitation by enabling biomechanically plausible, speed-flexible locomotion in simulation.

Abstract

Virtual models of human gait, or digital twins, offer a promising solution for studying mobility without the need for labor-intensive data collection. However, challenges such as the sim-to-real gap and limited adaptability to diverse walking conditions persist. To address these, we developed and validated a framework to create a skeletal humanoid agent capable of adapting to varying walking speeds while maintaining biomechanically realistic motions. The framework combines a synthetic data generator, which produces biomechanically plausible gait kinematics from open-source biomechanics data, and a training system that uses adversarial imitation learning to train the agent's walking policy. We conducted comprehensive analyses comparing the agent's kinematics, synthetic data, and the original biomechanics dataset. The agent achieved a root mean square error of 5.24 +- 0.09 degrees at varying speeds compared to ground-truth kinematics data, demonstrating its adaptability. This work represents a significant step toward developing a digital twin of human locomotion, with potential applications in biomechanics research, exoskeleton design, and rehabilitation.

Figures (6)

• Figure 1: Overview of our framework for training a speed-adaptive walking agent. The framework includes two main components: generating expert demonstrations and policy optimization. Expert demonstrations from motion data are used to train a discriminator that distinguishes expert- from policy-generated actions at different speeds. The reward function combines the discriminator's output with a speed reward (difference between the agent's target and actual center of mass speed). These rewards optimize the walking policy using trust region policy optimization. The target speed is part of the agent's observation space, with a progressive curriculum exposing the agent to a range of speeds during training. Evaluation tests the agent's ability to achieve the desired speed under varying conditions.
• Figure 2: Comparison of joint kinematics between open-source biomechanics and synthetic data across varying speeds. A linear model was fitted to joint kinematics from an open-source gait biomechanics dataset. This model was then used to generate synthetic kinematic profiles during training.
• Figure 3: Effect of curriculum learning on the walking agent's performance during training. The red plot represents the case for changing the target speed in a progressive way and the brown plot stands for a random case at every training epoch. Error bars indicate $\pm$1 standard deviation.
• Figure 4: Representative walking agent with optimal (top) and suboptimal (bottom) settings. Suboptimal agent (e.g., higher ratio of speed reward) exhibited abnormal gait patterns, such as irregular limb coordination, exaggerated dorsiflexion, and asymmetric range of motion.
• Figure 5: Walking agent performance with optimal and baseline settings. (A) Target speed tracking performance across varying conditions. Tracking error for (B) joint angles and (C) COM speed relative to ground-truth data. (D) Walking agent's adaptability to dynamically changing walking speeds. Error bars and shaded regions indicate $\pm$1 standard deviation, and asterisks indicate statistical significance ($\textit{p}<$ 0.05).