Online Optimization of Central Pattern Generators for Quadruped Locomotion

TL;DR

This paper presents a framework for the online optimization of the CPG parameters through Bayesian Optimization, and shows that this framework can rapidly optimize and adapt to varying velocity commands and changes in the terrain, for example to varying coefficients of friction, terrain slope angles, and added mass payloads placed on the robot.

Abstract

Typical legged locomotion controllers are designed or trained offline. This is in contrast to many animals, which are able to locomote at birth, and rapidly improve their locomotion skills with few real-world interactions. Such motor control is possible through oscillatory neural networks located in the spinal cord of vertebrates, known as Central Pattern Generators (CPGs). Models of the CPG have been widely used to generate locomotion skills in robotics, but can require extensive hand-tuning or offline optimization of inter-connected parameters with genetic algorithms. In this paper, we present a framework for the \textit{online} optimization of the CPG parameters through Bayesian Optimization. We show that our framework can rapidly optimize and adapt to varying velocity commands and changes in the terrain, for example to varying coefficients of friction, terrain slope angles, and added mass payloads placed on the robot. We study the effects of sensory feedback on the CPG, and find that both force feedback in the phase equations, as well as posture control (Virtual Model Control) are both beneficial for robot stability and energy efficiency. In hardware experiments on the Unitree Go1, we show rapid optimization (in under 3 minutes) and adaptation of energy-efficient gaits to varying target velocities in a variety of scenarios: varying coefficients of friction, added payloads up to 15 kg, and variable slopes up to 10 degrees. See demo at: https://youtu.be/4qq5leCI2AI

Figures (7)

• Figure 1: Optimized CPG-based locomotion tracking of $v_x^*$=0.4 m/s, rapidly adapting to various conditions, including normal flat terrain, slippery terrain (i.e. taped feet), a 15 kg payload, and a 10-degree slope.
• Figure 2: Control architecture for online CPG optimization. The optimizer updates the CPG parameters $\mathbf{x}$ at the beginning of each trial based on the current target velocity $v_x^*$ and contextual information $\mathbf{c}$. It receives the base velocity and joint data during the steady-state phase of each trial to compute the objective value. Force feedback to the CPG phase equations and Virtual Model Control help to stabilize the robot and promote energy-efficient gaits.
• Figure 3: Simulation results for online locomotion learning, where we report the mean forward velocity, Cost of Transport (CoT), objective value, and context information in different scenarios. (a): adapting to different coefficients of friction while tracking a target velocity of 0.5 m/s. (b): rapid adaptation to moving target velocities. (c): tracking 0.5 m/s with adaptation from flat terrain to varying slopes of 10 and 12.5 degrees. (d): tracking 0.5 m/s with adaptation to added loads of 7.5 kg and 15 kg. Red arrows indicate the transition time to a new contextual condition (slope or load).
• Figure 4: Slope adaptation and velocity tracking in hardware experiments. (a) mean velocity, CoT, objective value, and slope context, increasing the slope to 10 degrees while tracking 0.4 m/s. (b) optimized parameters evolution during (a), from flat terrain locomotion to slope adaptation. (c) adaptation to changing velocity commands (0.6$\rightarrow$0.3$\rightarrow$ 0.5$\rightarrow$0.6 m/s) on the hardware.
• Figure 5: Forward velocity during the hardware experiment of adaptation up to a 15 kg load, tracking a target velocity of 0.4 m/s. Before this adaptation, the robot was trained on flat terrain without any load for three minutes.