Replication of Impedance Identification Experiments on a Reinforcement-Learning-Controlled Digital Twin of Human Elbows

TL;DR

This work addresses the challenge of validating neuromechanical insights outside real human experiments by building a MyoSuite-based digital twin of the elbow controlled by a natural policy gradient RL agent. The authors reproduce classic impedance identification protocols (IE, MII, SII, DII) and estimate inertia and K-B-I impedance parameters, comparing RL-driven results to human data. Key findings show accurate inertia recovery and that the RL-controlled elbow exhibits higher impedance than humans, especially in static conditions, with dynamic tasks revealing RL stability despite perturbations. The study demonstrates the viability of virtual neuromechanical experiments for rapid, cost-effective rehabilitation research and outlines how such digital twins could inform experimental design and rehabilitation theory prior to human trials, with EMG-based analyses proposed for future work. The results imply that digital twins with RL control can serve as a valuable surrogate for preliminary validation and hypothesis testing in neuromechanics and robotic rehabilitation.

Abstract

This study presents a pioneering effort to replicate human neuromechanical experiments within a virtual environment utilising a digital human model. By employing MyoSuite, a state-of-the-art human motion simulation platform enhanced by Reinforcement Learning (RL), multiple types of impedance identification experiments of human elbow were replicated on a musculoskeletal model. We compared the elbow movement controlled by an RL agent with the motion of an actual human elbow in terms of the impedance identified in torque-perturbation experiments. The findings reveal that the RL agent exhibits higher elbow impedance to stabilise the target elbow motion under perturbation than a human does, likely due to its shorter reaction time and superior sensory capabilities. This study serves as a preliminary exploration into the potential of virtual environment simulations for neuromechanical research, offering an initial yet promising alternative to conventional experimental approaches. An RL-controlled digital twin with complete musculoskeletal models of the human body is expected to be useful in designing experiments and validating rehabilitation theory before experiments on real human subjects.

Figures (6)

• Figure 1: Framework of comparing the neuromechanical properties between the RL-controlled elbow model and human subjects through the impedance identification experiments
• Figure 2: Neuromechanical characteristics of a virual or real elbow's responses to external perturbations can be described as an impedance model. Left: musculoskeletal model of an elbow in the MyoSuite from the top view. Middle: K-B-I impedance model of an elbow joint. Right: apparatus and protocol from Popescu et al.'s study on elbow impedance popescu2003elbow ($\tau$ is the applied torque perturbation; $\theta$ is the elbow joint angle; $K$ is the joint stiffness; $B$ is the joint viscosity; $I$ is the moment of inertia of the forearm).
• Figure 3: Estimation of the moment of inertia based on the MyoSuite elbow model; the green line with dots is the estimate results in the simulation experiments; the red line is the ground truth calculated according to the model parameter
• Figure 4: Resulting joint angle trace in an example of the MII experiment and corresponding predicted trace generated by the identified K-B-I model.
• Figure 5: Elbow stiffness and viscosity of the human and RL agent in the static condition; the green points are the identification results of the SII experiment; the red dashed lines of non-agent are the identification results of the MII experiment; the slash-and-dash purple lines are the experimental results from human subjects popescu2003elbow.