Reinforcement Learning with Adaptive Curriculum Dynamics Randomization for Fault-Tolerant Robot Control

TL;DR

An adaptive curriculum reinforcement learning algorithm with dynamics randomization (ACDR) is established that can adaptively train a quadruped robot in random actuator failure conditions and formulate a single robust policy for fault-tolerant robot control.

Abstract

This study is aimed at addressing the problem of fault tolerance of quadruped robots to actuator failure, which is critical for robots operating in remote or extreme environments. In particular, an adaptive curriculum reinforcement learning algorithm with dynamics randomization (ACDR) is established. The ACDR algorithm can adaptively train a quadruped robot in random actuator failure conditions and formulate a single robust policy for fault-tolerant robot control. It is noted that the hard2easy curriculum is more effective than the easy2hard curriculum for quadruped robot locomotion. The ACDR algorithm can be used to build a robot system that does not require additional modules for detecting actuator failures and switching policies. Experimental results show that the ACDR algorithm outperforms conventional algorithms in terms of the average reward and walking distance.

Figures (10)

• Figure 1: Illustration of quadruped robot locomotion with actuator failure. (Top) A quadruped robot trained by basic reinforcement learning can exhibit locomotion. (Bottom) The quadruped robot turns over if the actuator of a leg (red) fails.
• Figure 2: Time update of intervals $[L,U]$ of the failure coefficient $k$ in curriculum learning. The intervals are represented by the colored regions. ACDR_e2h and ACDR_h2e represent easy2hard and hard2easy curricula for ACDR, respectively. LCDR_e2h and LCDR_h2h represent linear easy2hard and hard2easy curricula, respectively.
• Figure 3: Average reward for the plain (blue) and broken (green) quadruped tasks. Error bars indicate the standard error.
• Figure 4: Average progress for the plain (blue) and broken (green) quadruped tasks. Error bars indicate the standard error.
• Figure 5: Average reward of all algorithms over various failure coefficients $k\in[0,1]$.