Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Legged Locomotion via Online Learning for Model Predictive Control

Hongyu Zhou, Xiaoyu Zhang, Vasileios Tzoumas

TL;DR

Adaptive Legged Locomotion via Online Learning and Model Predictive Control presents a real-time framework that learns residual dynamics online and integrates them into MPC for quadruped locomotion under unknown disturbances. It uses random Fourier features to represent residuals in an RKHS and online least-squares for model updates, achieving sublinear dynamic regret. Validation in Gazebo and MuJoCo under constant and time-varying disturbances demonstrates significant improvements in tracking over nominal MPC and L1-MPC, even with payloads up to 8 kg and rough terrains. The work advances adaptive legged control by enabling one-shot online adaptation without offline training and providing finite-time regret guarantees.

Abstract

We provide an algorithm for adaptive legged locomotion via online learning and model predictive control. The algorithm is composed of two interacting modules: model predictive control (MPC) and online learning of residual dynamics. The residual dynamics can represent modeling errors and external disturbances. We are motivated by the future of autonomy where quadrupeds will autonomously perform complex tasks despite real-world unknown uncertainty, such as unknown payload and uneven terrains. The algorithm uses random Fourier features to approximate the residual dynamics in reproducing kernel Hilbert spaces. Then, it employs MPC based on the current learned model of the residual dynamics. The model is updated online in a self-supervised manner using least squares based on the data collected while controlling the quadruped. The algorithm enjoys sublinear \textit{dynamic regret}, defined as the suboptimality against an optimal clairvoyant controller that knows how the residual dynamics. We validate our algorithm in Gazebo and MuJoCo simulations, where the quadruped aims to track reference trajectories. The Gazebo simulations include constant unknown external forces up to $12\boldsymbol{g}$, where $\boldsymbol{g}$ is the gravity vector, in flat terrain, slope terrain with $20\degree$ inclination, and rough terrain with $0.25m$ height variation. The MuJoCo simulations include time-varying unknown disturbances with payload up to $8~kg$ and time-varying ground friction coefficients in flat terrain.

Adaptive Legged Locomotion via Online Learning for Model Predictive Control

TL;DR

Adaptive Legged Locomotion via Online Learning and Model Predictive Control presents a real-time framework that learns residual dynamics online and integrates them into MPC for quadruped locomotion under unknown disturbances. It uses random Fourier features to represent residuals in an RKHS and online least-squares for model updates, achieving sublinear dynamic regret. Validation in Gazebo and MuJoCo under constant and time-varying disturbances demonstrates significant improvements in tracking over nominal MPC and L1-MPC, even with payloads up to 8 kg and rough terrains. The work advances adaptive legged control by enabling one-shot online adaptation without offline training and providing finite-time regret guarantees.

Abstract

We provide an algorithm for adaptive legged locomotion via online learning and model predictive control. The algorithm is composed of two interacting modules: model predictive control (MPC) and online learning of residual dynamics. The residual dynamics can represent modeling errors and external disturbances. We are motivated by the future of autonomy where quadrupeds will autonomously perform complex tasks despite real-world unknown uncertainty, such as unknown payload and uneven terrains. The algorithm uses random Fourier features to approximate the residual dynamics in reproducing kernel Hilbert spaces. Then, it employs MPC based on the current learned model of the residual dynamics. The model is updated online in a self-supervised manner using least squares based on the data collected while controlling the quadruped. The algorithm enjoys sublinear \textit{dynamic regret}, defined as the suboptimality against an optimal clairvoyant controller that knows how the residual dynamics. We validate our algorithm in Gazebo and MuJoCo simulations, where the quadruped aims to track reference trajectories. The Gazebo simulations include constant unknown external forces up to , where is the gravity vector, in flat terrain, slope terrain with inclination, and rough terrain with height variation. The MuJoCo simulations include time-varying unknown disturbances with payload up to and time-varying ground friction coefficients in flat terrain.
Paper Structure (14 sections, 3 theorems, 9 equations, 5 figures, 1 table)

This paper contains 14 sections, 3 theorems, 9 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adaptive Legged Locomotion via Online Learning and Model Predictive Control
  4. Algorithm and Regret Guarantee
  5. Function Approximation via Random Fourier Features
  6. Online Least-Squares Estimation
  7. Algorithm for \ref{['prob:control']}
  8. No-Regret Guarantee
  9. Numerical Experiments
  10. Gazebo Simulations
  11. MuJoCo Simulations
  12. Failure Cases
  13. Discussion on Real-Time Computation
  14. Conclusion

Key Result

Proposition 1

Assume $\boldsymbol{h} \in$$\mathcal{F}_{2}\left(B_{{h}}\right)$, where Let $\delta \in(0,1)$. With probability at least $1-\delta$, there exist $\left\{\boldsymbol{\alpha}_{i}\right\}_{i=1}^{M} \in {\cal D}$, i.e., $\|\boldsymbol{\alpha}_{i}\| \leq B_h$, such that

Figures (5)

  • Figure 1: Architecture of Adaptive Legged Locomotion via Online Learning and Model Predictive Control. The pipeline is composed of two modules: (i) a model predictive control (MPC) module, and (ii) an online learning module. The MPC module uses the learned residual dynamics model from the online learning module to calculate the next control input. Given the control input and the observed new state, the online learning module then updates the residual dynamics model.
  • Figure 2: Sample trajectories of the Gazebo Simulations in \ref{['subsec:sim-gazebo']}. Three scenarios in flat, slope, and rough terrains are provided. Our method enables the quadruped to reach the goal position while the nominal MPC fails. We plot our method and L1-MPC separately in the case of rough terrain since they move forward at different speeds due to $x$-direction forces and results in different reference trajectories in $x$ and $z$. (a) and (b): The nominal MPC fails due to the heavy load. (c) and (d): The nominal MPC fails to move forward due to the $x$-component of $\boldsymbol{f}_{u}$. Our method exhibits faster response to $\boldsymbol{f}_{\boldsymbol{u}}$ than L1-MPC. (a) and (b): faster tracking in $z$-direction, (c) and (d): faster tracking in $x$-direction despite the same velocity command, i.e., ours reaches $x=6~m$ faster.
  • Figure 3: Learned residual forces and torques $\hat{\boldsymbol{h}}\left(\boldsymbol{z}_{t};\hat{\boldsymbol{\alpha}}_t\right)$ over flat terrain with different $\boldsymbol{f}_u$ in \ref{['subsec:sim-gazebo']}. The main residual dynamics come from the force in $z$-direction, to which $\hat{\boldsymbol{h}}\left(\boldsymbol{z}_{t};\hat{\boldsymbol{\alpha}}_t\right)$ converges as the online learning module collects more data on-the-fly. This demonstrates that the online learning module is able to adapt to both constant and time-varying residual dynamics.
  • Figure 4: Sample trajectories of the MuJoCo Experiments with constant friction coefficient in \ref{['subsec:sim-mujoco']}. Scenarios with no payload, $4~kg$, and $8~kg$ payload are provided. Our method enables the quadruped to track the $0.5~m/s$ velocity command while maintaining the $0.3~m$ body height under different payload conditions. While the Nominal MPC has larger tracking errors in $z$ and $v_x$, and fails under $8~kg$ payload.
  • Figure 5: Sample trajectories of the MuJoCo Experiments with varying friction coefficients in \ref{['subsec:sim-mujoco']}. Scenarios with no payload, $4~kg$, and $8~kg$ payload are provided. Our method enables the quadruped to track the $0.5~m/s$ velocity command while maintaining the $0.3~m$ body height under different payload conditions. While the Nominal MPC has larger tracking errors in $z$ and $v_x$, and fails under $8~kg$ payload.

Theorems & Definitions (4)

  • Definition 1: Dynamic Regret
  • Proposition 1: Uniformly Approximation Error boffi2022nonparametric
  • Proposition 2: Regret Bound of Online Least-Squares Estimation hazan2016introduction
  • Theorem 1: Dynamic Regret Guarantee zhou2024simultaneous