Adaptive Torque Control of Exoskeletons under Spasticity Conditions via Reinforcement Learning

TL;DR

<3-5 sentence high-level summary>The paper tackles safety and personalization in wearable knee exoskeletons for people with spasticity, where velocity-dependent reflexes alter dynamics. It introduces a digital twin combining musculoskeletal and exoskeleton models with a differentiable spasticity model. An adaptive torque controller based on Soft Actor-Critic learns to mitigate spasticity effects and reduce interaction torques compared with a PID baseline. Results show average knee torque reductions of about 10.6% and RMS reductions of about 8.9%, supporting the viability of RL for safe, subject-specific WR control.

Abstract

Spasticity is a common movement disorder symptom in individuals with cerebral palsy, hereditary spastic paraplegia, spinal cord injury and stroke, being one of the most disabling features in the progression of these diseases. Despite the potential benefit of using wearable robots to treat spasticity, their use is not currently recommended to subjects with a level of spasticity above ${1^+}$ on the Modified Ashworth Scale. The varying dynamics of this velocity-dependent tonic stretch reflex make it difficult to deploy safe personalized controllers. Here, we describe a novel adaptive torque controller via deep reinforcement learning (RL) for a knee exoskeleton under joint spasticity conditions, which accounts for task performance and interaction forces reduction. To train the RL agent, we developed a digital twin, including a musculoskeletal-exoskeleton system with joint misalignment and a differentiable spastic reflexes model for the muscles activation. Results for a simulated knee extension movement showed that the agent learns to control the exoskeleton for individuals with different levels of spasticity. The proposed controller was able to reduce maximum torques applied to the human joint under spastic conditions by an average of 10.6\% and decreases the root mean square until the settling time by 8.9\% compared to a conventional compliant controller.

Figures (7)

• Figure 1: RL Exo Pipeline. Digital twin development with MyoexoLeg system and spastic model in a red box at left. Deep RL architecture in simulation with RL algorithm and environment in orange box at rigth. The interactions between the digital twin and Deep RL architecture are presented at the middle.
• Figure 2: Knee-Exo coupling model with the upper Exo in red and lower Exo in grey. a) Front view of the Human-Exo attachment, including passive joints in black and actuated joints in red. b) Lateral view of the Slider-Crank mechanism designed to simulate a soft knee-Exo coupling, with angular joints ranges for thigh's and shank's cranks in red and thigh's slider range in blue.
• Figure 3: Knee flexion-extension and spasticity model. (a) Spastic regions (yellow), knee joint extension movement (red). (b) Spasticity coefficients as a function of the knee angle (top) and velocity (bottom) for each spasticity level including individual variance to account for inter-subject variability. Level 0 (green), level 1 (yellow), level 2 (orange) and level 3 (red).
• Figure 4: Simulation results of the SAC algorithm for the implemented spasticity levels. In green, yellow, orange and red for level 0 to 3. The upper graph shows the joint position error, the middle graph, the control signal and the lower graphs shows the interaction torques, during 8s of simulation.
• Figure 5: Comparison of PID and SAC algorithms for different levels of spasticity, grouped in columns. In black, PID; in yellow, SAC, representing their mean and standard deviation values. From left to right in columns, the levels of spasticity implemented from a) Level 0 b) Level 1 c) Level 2 d) Level 3. From top to bottom the comparison of a) Position error b) Control signal c) Interaction torque summary exerted by the exoskeleton and spastic reflexes.