Exoskeleton Control through Learning to Reduce Biological Joint Moments in Simulations

TL;DR

An RL framework to learn exoskeleton assistance policies that reduce biological joint moments, and a validation pipeline that verifies the trained control networks using an open-source gait dataset through inference and comparison with biological joint moments are presented.

Abstract

Data-driven joint-moment predictors offer a scalable alternative to laboratory-based inverse-dynamics pipelines for biomechanics estimation and exoskeleton control. Meanwhile, physics-based reinforcement learning (RL) enables simulation-trained controllers to learn dynamics-aware assistance strategies without extensive human experimentation. However, quantitative verification of simulation-trained exoskeleton torque predictors, and their impact on human joint power injection, remains limited. This paper presents (1) an RL framework to learn exoskeleton assistance policies that reduce biological joint moments, and (2) a validation pipeline that verifies the trained control networks using an open-source gait dataset through inference and comparison with biological joint moments. Simulation-trained multilayer perceptron (MLP) controllers are developed for level-ground and ramp walking, mapping short-horizon histories of bilateral hip and knee kinematics to normalized assistance torques. Results show that predicted assistance preserves task-intensity trends across speeds and inclines. Agreement is particularly strong at the hip, with cross-correlation coefficients reaching 0.94 at 1.8 m/s and 0.98 during 5° decline walking, demonstrating near-matched temporal structure. Discrepancies increase at higher speeds and steeper inclines, especially at the knee, and are more pronounced in joint power comparisons. Delay tuning biases assistance toward greater positive power injection; modest timing shifts increase positive power and improve agreement in specific gait intervals. Together, these results establish a quantitative validation framework for simulation-trained exoskeleton controllers, demonstrate strong sim-to-data consistency at the torque level, and highlight both the promise and the remaining challenges for sim-to-real transfer.

Figures (7)

• Figure 1: The RL learning framework with Exoskeleton Control Network (ECN) and Muscle Coordination Network (MCN). Idealized torque assistance (blue curved arrows) is used instead of modeling any specific physical devices. The Human Control Network (HCN) predicted action is transferred to the desired torques (i.e. human biological joint moments), which is affected by exoskeleton assistance and corresponds to the full total joint torque minus the assistance. The desired biological torques are fed to MCN to predict muscle activations. The ECN predicts normalized assistance torques and is trained in supervised learning to reduce the biological joint moments via the loss in Eq. \ref{['eq:loss']}. The trained ECN and MCN work together to control the human model for walking on level ground and ramps.
• Figure 2: Simulation results of walking gait on (a)level ground, (b)inclined (10°) and (c)declined (10°) grade surfaces with idealized hip and knee assistance. The purple lines on the foot are ground reaction forces.
• Figure 3: Inference hip and knee assistance torque and power (Pred) vs. ground truth (GT) biological moment and power under level-ground walking. The sign of hip and knee torques: hip flexion is negative and knee flexion is positive.
• Figure 4: Inference hip and knee assistance torque and power vs. GT biological moment and power under ramp walking. Positive degree correspond to incline walking, negative degree denotes decline walking, and 1.2 m/s represents the reference level-ground walking.
• Figure 5: Inference hip and knee assistance torque and power generated from the level-ground walking model vs. GT biological moment and power under ramp walking. This is a mismatched model-task condition to examine the generalization capability of the level-ground walking model.