Real-Time Gait Adaptation for Quadrupeds using Model Predictive Control and Reinforcement Learning

TL;DR

The authors address the limitation of fixed gait policies in quadrupedal locomotion by introducing a hybrid framework that couples Model Predictive Path Integral (MPPI) planning with a Dreamer-based learned model to enable real-time, continuous gait adaptation. Gait parameters are optimized jointly with actions in an online optimization, guided by a generalized reward for deployment and a terminal value for infinite-horizon planning. The approach demonstrates substantial energy efficiency gains (up to 36.48% across speeds) and smooth, task-appropriate gait transitions in simulation on the Unitree Go1, with a modular architecture that supports terrain-aware extensions. This work advances practical quadruped locomotion by unifying gait versatility, stability, and energy efficiency through real-time planning and learned priors.

Abstract

Model-free reinforcement learning (RL) has enabled adaptable and agile quadruped locomotion; however, policies often converge to a single gait, leading to suboptimal performance. Traditionally, Model Predictive Control (MPC) has been extensively used to obtain task-specific optimal policies but lacks the ability to adapt to varying environments. To address these limitations, we propose an optimization framework for real-time gait adaptation in a continuous gait space, combining the Model Predictive Path Integral (MPPI) algorithm with a Dreamer module to produce adaptive and optimal policies for quadruped locomotion. At each time step, MPPI jointly optimizes the actions and gait variables using a learned Dreamer reward that promotes velocity tracking, energy efficiency, stability, and smooth transitions, while penalizing abrupt gait changes. A learned value function is incorporated as terminal reward, extending the formulation to an infinite-horizon planner. We evaluate our framework in simulation on the Unitree Go1, demonstrating an average reduction of up to 36.48 % in energy consumption across varying target speeds, while maintaining accurate tracking and adaptive, task-appropriate gaits.

Figures (5)

• Figure 1: Conventional approaches (left) rely on discrete gait switching with predefined libraries. Our framework (right) performs joint optimization over actions and continuous gait parameters using learned dynamics and a reward-based planner.
• Figure 2: Overview of the training pipeline: the dreamer module learns latent dynamics, reward, and value functions from observation histories, while the expert actor is trained using supervised signals from dreamer outputs.
• Figure 3: Pictorial representation of the MPPI algorithm: the warm start utilises dreamer policy and dynamics to generate initial trajectories to help MPPI.
• Figure 4: Performance comparison of fixed gaits across command velocities on flat terrain. The results illustrate how linear and angular tracking errors, as well as cost of transport, vary significantly with gait choice—especially at higher velocities—highlighting the limitations of using a single gait policy and the need for adaptive gait selection.
• Figure 5: Evaluation of the proposed locomotion planner under a continuously increasing velocity command. The robot effectively tracks the desired forward velocity with smooth acceleration. Joint trajectories exhibit stable and periodic behavior, indicating coordinated motion. Mean power consumption shows an initial peak during acceleration followed by steady oscillations. The gait diagram illustrates consistent foot contact patterns and adaptive gait transitions corresponding to the increasing command speed, with representative snapshots shown below.