Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement-Learning-Based Assistance Reduces Squat Effort with a Modular Hip--Knee Exoskeleton

Neethan Ratnakumar, Mariya Huzaifa Tohfafarosh, Saanya Jauhri, Xianlian Zhou

Abstract

Squatting is one of the most demanding lower-limb movements, requiring substantial muscular effort and coordination. Reducing the physical demands of this task through intelligent and personalized assistance has significant implications, particularly in industries involving repetitive low-level assembly activities. In this study, we evaluated the effectiveness of a neural network controller for a modular Hip-Knee exoskeleton designed to assist squatting tasks. The neural network controller was trained via reinforcement learning (RL) in a physics-based, human-exoskeleton interaction simulation environment. The controller generated real-time hip and knee assistance torques based on recent joint-angle and velocity histories. Five healthy adults performed three-minute metronome-guided squats under three conditions: (1) no exoskeleton (No-Exo), (2) exoskeleton with Zero-Torque, and (3) exoskeleton with active assistance (Assistance). Physiological effort was assessed using indirect calorimetry and heart rate monitoring, alongside concurrent kinematic data collection. Results show that the RL-based controller adapts to individuals by producing torque profiles tailored to each subject's kinematics and timing. Compared with the Zero-Torque and No-Exo condition, active assistance reduced the net metabolic rate by approximately 10%, with minor reductions observed in heart rate. However, assisted trials also exhibited reduced squat depth, reflected by smaller hip and knee flexion. These preliminary findings suggest that the proposed controller can effectively lower physiological effort during repetitive squatting, motivating further improvements in both hardware design and control strategies.

Reinforcement-Learning-Based Assistance Reduces Squat Effort with a Modular Hip--Knee Exoskeleton

Abstract

Squatting is one of the most demanding lower-limb movements, requiring substantial muscular effort and coordination. Reducing the physical demands of this task through intelligent and personalized assistance has significant implications, particularly in industries involving repetitive low-level assembly activities. In this study, we evaluated the effectiveness of a neural network controller for a modular Hip-Knee exoskeleton designed to assist squatting tasks. The neural network controller was trained via reinforcement learning (RL) in a physics-based, human-exoskeleton interaction simulation environment. The controller generated real-time hip and knee assistance torques based on recent joint-angle and velocity histories. Five healthy adults performed three-minute metronome-guided squats under three conditions: (1) no exoskeleton (No-Exo), (2) exoskeleton with Zero-Torque, and (3) exoskeleton with active assistance (Assistance). Physiological effort was assessed using indirect calorimetry and heart rate monitoring, alongside concurrent kinematic data collection. Results show that the RL-based controller adapts to individuals by producing torque profiles tailored to each subject's kinematics and timing. Compared with the Zero-Torque and No-Exo condition, active assistance reduced the net metabolic rate by approximately 10%, with minor reductions observed in heart rate. However, assisted trials also exhibited reduced squat depth, reflected by smaller hip and knee flexion. These preliminary findings suggest that the proposed controller can effectively lower physiological effort during repetitive squatting, motivating further improvements in both hardware design and control strategies.
Paper Structure (10 sections, 4 equations, 9 figures, 1 table)

This paper contains 10 sections, 4 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Exoskeleton Hardware
  4. Reinforcement Learning Training Framework for Squatting Assistance
  5. Musculoskeletal model
  6. Experimental Protocol
  7. Data Processing and Statistics
  8. Results
  9. Discussion
  10. Conclusions

Figures (9)

  • Figure 1: Modular hip--knee exoskeleton design showing front, side, and back views.
  • Figure 2: The RL learning framework with Exoskeleton Control Network (ECN) and Muscle Coordination Network (MCN). Idealized torque assistance (green curved arrows) is used instead of modeling the human-exoskeleton interaction. The human Control Policy Network (CPN) is not presented here for brevity. The ECN predicts a normalized torque that is scaled by the maximum torque. The RL trained ECN and MCN work together to control the human model squatting up and down.
  • Figure 3: A representative participant wearing the modular hip--knee exoskeleton and metabolic measurement mask, shown from the front, side, and back.
  • Figure 4: Representative participant performing a squat cycle with the exoskeleton.
  • Figure 5: Heart Rate (bpm) by Condition
  • ...and 4 more figures