Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning

Tan-Hanh Pham, Godwyll Aikins, Tri Truong, Kim-Doang Nguyen

TL;DR

This study develops a reinforcement learning (RL) framework to adaptively compensate for a nonfunctional joint during task execution, and shows that the RL algorithm enables the robot to successfully complete tasks even with joint failures, achieving a high success rate.

Abstract

Robotic manipulators are widely used in various industries for complex and repetitive tasks. However, they remain vulnerable to unexpected hardware failures. In this study, we address the challenge of enabling a robotic manipulator to complete tasks despite joint malfunctions. Specifically, we develop a reinforcement learning (RL) framework to adaptively compensate for a non-functional joint during task execution. Our experimental platform is the Franka robot with 7 degrees of freedom (DOFs). We formulate the problem as a partially observable Markov decision process (POMDP), where the robot is trained under various joint failure conditions and tested in both seen and unseen scenarios. We consider scenarios where a joint is permanently broken and where it functions intermittently. Additionally, we demonstrate the effectiveness of our approach by comparing it with traditional inverse kinematics-based control methods. The results show that the RL algorithm enables the robot to successfully complete tasks even with joint failures, achieving a high success rate with an average rate of 93.6%. This showcases its robustness and adaptability. Our findings highlight the potential of RL to enhance the resilience and reliability of robotic systems, making them better suited for unpredictable environments. All related codes and models are published online.

Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning

TL;DR

This study develops a reinforcement learning (RL) framework to adaptively compensate for a nonfunctional joint during task execution, and shows that the RL algorithm enables the robot to successfully complete tasks even with joint failures, achieving a high success rate.

Abstract

Robotic manipulators are widely used in various industries for complex and repetitive tasks. However, they remain vulnerable to unexpected hardware failures. In this study, we address the challenge of enabling a robotic manipulator to complete tasks despite joint malfunctions. Specifically, we develop a reinforcement learning (RL) framework to adaptively compensate for a non-functional joint during task execution. Our experimental platform is the Franka robot with 7 degrees of freedom (DOFs). We formulate the problem as a partially observable Markov decision process (POMDP), where the robot is trained under various joint failure conditions and tested in both seen and unseen scenarios. We consider scenarios where a joint is permanently broken and where it functions intermittently. Additionally, we demonstrate the effectiveness of our approach by comparing it with traditional inverse kinematics-based control methods. The results show that the RL algorithm enables the robot to successfully complete tasks even with joint failures, achieving a high success rate with an average rate of 93.6%. This showcases its robustness and adaptability. Our findings highlight the potential of RL to enhance the resilience and reliability of robotic systems, making them better suited for unpredictable environments. All related codes and models are published online.
Paper Structure (18 sections, 23 equations, 4 figures, 3 tables)

This paper contains 18 sections, 23 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. Problem Formulation
  5. Observation Space
  6. Action Space
  7. Reward Function
  8. Reinforcement Learning Framework
  9. PPO Algorithm
  10. Agent Training
  11. Simulation and Experiment Setup
  12. Inverse Kinematics
  13. Denavit-Hartenberg Parameters and Inverse Kinematics Solving
  14. Experiment Setup and Joint Malfunction
  15. Results and Discussion
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Our framework addresses the issue of joint malfunction in robot manipulators using the PPO algorithm. The Actor takes in the observation from the environment and outputs action for joints. The critic estimates the value of a state, helping the actor learn by providing feedback on the quality of its actions.
  • Figure 2: Reward from the training
  • Figure 3: Example of successful task completion and failed task completion.
  • Figure 4: Comparison of the trajectories of the end-effector when one of the joints of the robot is broken and when all the joints work properly. Typically, when the robot operates properly, it must follow the desired trajectory (orange lines) constructed by three joints: the initial position ($p_i$), the drawer position ($p_d$), and the pulled-out position ($p_p$).