Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human-Machine Co-Adaptation for Robot-Assisted Rehabilitation via Dual-Agent Multiple Model Reinforcement Learning (DAMMRL)

Yang An, Yaqi Li, Hongwei Wang, Rob Duffield, Steven W. Su

TL;DR

The paper tackles the challenge of human-machine co-adaptation in robot-assisted ankle rehabilitation by introducing the Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework. It combines a cooperative, dual-agent MDP approach with a multiple-model RL strategy to simplify action spaces and enable online switching among sub-models, using neural networks to learn switching policies. The system integrates PD/PID sub-controllers, VR/AR human–machine interfaces, and offline Simulink-based training with real-time testing, validated on 13 healthy subjects across multiple reward-structure settings. Key findings show that reward design and sub-model selection critically shape active participation, tracking performance, and overall rehabilitation efficacy, indicating DAMMRL’s potential for patient-specific, adaptive therapy while highlighting the path toward improved explainability through model reduction. The framework advances adaptive, patient-centered therapeutic interventions by enabling co-adaptive control that balances safety, efficacy, and user engagement in both simulated and real-world environments.

Abstract

This study introduces a novel approach to robot-assisted ankle rehabilitation by proposing a Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework, leveraging multiple model adaptive control (MMAC) and co-adaptive control strategies. In robot-assisted rehabilitation, one of the key challenges is modelling human behaviour due to the complexity of human cognition and physiological systems. Traditional single-model approaches often fail to capture the dynamics of human-machine interactions. Our research employs a multiple model strategy, using simple sub-models to approximate complex human responses during rehabilitation tasks, tailored to varying levels of patient incapacity. The proposed system's versatility is demonstrated in real experiments and simulated environments. Feasibility and potential were evaluated with 13 healthy young subjects, yielding promising results that affirm the anticipated benefits of the approach. This study not only introduces a new paradigm for robot-assisted ankle rehabilitation but also opens the way for future research in adaptive, patient-centred therapeutic interventions.

Human-Machine Co-Adaptation for Robot-Assisted Rehabilitation via Dual-Agent Multiple Model Reinforcement Learning (DAMMRL)

TL;DR

The paper tackles the challenge of human-machine co-adaptation in robot-assisted ankle rehabilitation by introducing the Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework. It combines a cooperative, dual-agent MDP approach with a multiple-model RL strategy to simplify action spaces and enable online switching among sub-models, using neural networks to learn switching policies. The system integrates PD/PID sub-controllers, VR/AR human–machine interfaces, and offline Simulink-based training with real-time testing, validated on 13 healthy subjects across multiple reward-structure settings. Key findings show that reward design and sub-model selection critically shape active participation, tracking performance, and overall rehabilitation efficacy, indicating DAMMRL’s potential for patient-specific, adaptive therapy while highlighting the path toward improved explainability through model reduction. The framework advances adaptive, patient-centered therapeutic interventions by enabling co-adaptive control that balances safety, efficacy, and user engagement in both simulated and real-world environments.

Abstract

This study introduces a novel approach to robot-assisted ankle rehabilitation by proposing a Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework, leveraging multiple model adaptive control (MMAC) and co-adaptive control strategies. In robot-assisted rehabilitation, one of the key challenges is modelling human behaviour due to the complexity of human cognition and physiological systems. Traditional single-model approaches often fail to capture the dynamics of human-machine interactions. Our research employs a multiple model strategy, using simple sub-models to approximate complex human responses during rehabilitation tasks, tailored to varying levels of patient incapacity. The proposed system's versatility is demonstrated in real experiments and simulated environments. Feasibility and potential were evaluated with 13 healthy young subjects, yielding promising results that affirm the anticipated benefits of the approach. This study not only introduces a new paradigm for robot-assisted ankle rehabilitation but also opens the way for future research in adaptive, patient-centred therapeutic interventions.
Paper Structure (17 sections, 13 equations, 35 figures, 3 tables)

This paper contains 17 sections, 13 equations, 35 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. Settings of Simulation training Environment and real-time testing environment
  3. The overall structure of the system
  4. The Key Components of Dual Agent RL
  5. The Practical Structure of Multiple-Model RL
  6. Network Structures of the Two Agents
  7. Details about the construction of reward function
  8. Training the agents
  9. The arrangements of the multiple models for both human and machine agents
  10. Reward Functions
  11. Sub-Controllers
  12. Settings and Parameters
  13. Analysis
  14. The Training and Experimental Procedure
  15. Experiments
  16. ...and 2 more sections

Figures (35)

  • Figure 1: The robot's model in Simulink/SimMechanics used for RL training.
  • Figure 2: Matlab Simulink and Python integrated structure via serial port wireless communication.
  • Figure 3: Visual stimulation in Robot-Assisted Rehabilitation
  • Figure 4: Ankle rehabilitation robot developed at the Jinan Key Lab of Intelligent Rehabilitation Robotics.
  • Figure 5: The block diagram of the training and testing environments.
  • ...and 30 more figures