AllGaits: Learning All Quadruped Gaits and Transitions

TL;DR

While the currently most popular gait (trot) does not result in the lowest COT, it is found that considering different co-dependent metrics such as mean base angular velocity and joint acceleration result in different 'optimal' gaits than those that minimize COT.

Abstract

We present a framework for learning a single policy capable of producing all quadruped gaits and transitions. The framework consists of a policy trained with deep reinforcement learning (DRL) to modulate the parameters of a system of abstract oscillators (i.e. Central Pattern Generator), whose output is mapped to joint commands through a pattern formation layer that sets the gait style, i.e. body height, swing foot ground clearance height, and foot offset. Different gaits are formed by changing the coupling between different oscillators, which can be instantaneously selected at any velocity by a user. With this framework, we systematically investigate which gait should be used at which velocity, and when gait transitions should occur from a Cost of Transport (COT), i.e. energy-efficiency, point of view. Additionally, we note how gait style changes as a function of locomotion speed for each gait to keep the most energy-efficient locomotion. While the currently most popular gait (trot) does not result in the lowest COT, we find that considering different co-dependent metrics such as mean base velocity and joint acceleration result in different `optimal' gaits than those that minimize COT. We deploy our controller in various hardware experiments, showing all 9 typical quadruped animal gaits, and demonstrate generalizability to unseen gaits during training, and robustness to leg failures. Video results can be found at https://youtu.be/OLoWSX_R868.

Figures (7)

• Figure 1: AllGaits: snapshots from learning all quadruped gaits with Central Pattern Generators and deep reinforcement learning.
• Figure 2: AllGaits: Control architecture for learning central pattern generators to locomote at all gaits for quadruped robots. The observation consists of velocity commands, proprioceptive measurements, and the current CPG states (efference copy of the spinal cord), which the policy network uses to select CPG parameters $\mu$ and $\omega$ for each leg $i$ (Front Right (FR), Front Left (FL), Hind Right (HR), Hind Left (HL)) to coordinate the Rhythm Generation. A gait coupling matrix is input from the user to set a particular gait. The resulting CPG states are then mapped to desired foot positions in a Pattern Formation layer, which the user can also directly modulate by setting body height $h$, swing foot ground clearance $g_c$, and foot offset from the hip $x_{off}$. This task-space mapping is then converted to desired joint angles with inverse kinematics, and finally tracked with joint PD control to produce torques $\bm{\tau}$. The control policy selects actions at 100 Hz, and all other blocks operate at 1 kHz.
• Figure 3: Contact timing for each foot with the ground as a percentage of a single gait cycle for various quadruped gaits: Lateral Sequence Walk, Amble, Trot, Pace, Bound, Pronk, Canter, Transverse Gallop (T.G.), Rotary Gallop (R.G.). These timings are converted to matrices that denote phase offsets between different limbs in column order: Front Right (FR), Front Left (FL), Hind Right (HR), Hind Left (HL), as they appear in Equation \ref{['eq:salamander_theta']}.
• Figure 4: Effects on the Cost of Transport for all gaits from modulating nominal (A) body height, and (B) foot offset relative to the hip around which oscillations occur. While the policy is capable of locomotion with all of these gait styles (with notable difficulty for the pronk gait with foot offset 0.025 $m$ in front of the hip above 2 $m/s$), each of these parameters significantly affects the COT, and different combinations of these result in better energy-efficiency at different velocities, most obviously with respect to the foot offset in (B).
• Figure 5: Most optimal Cost of Transport (COT) for all quadruped gaits, and corresponding gait style parameters for all velocities with our framework. (A): minimum COT possible for each gait. Walk is most optimal for velocities below 0.9 $m/s$, and pace is most optimal for higher velocities. (B): corresponding gait style parameters for the minimum COTs in (A), with varying body heights $h$ and foot offsets $x_{off}$ for all gaits. The lowest COTs here were all achieved with the lowest swing foot ground clearance $g_c = 0.02\ m$.