Perceptive Locomotion through Whole-Body MPC and Optimal Region Selection

TL;DR

The paper tackles real-time perceptive locomotion for legged robots by decoupling surface selection from whole-body MPC. It introduces a surface-repositioning module and a complete pipeline that integrates elevation-map-based perception, convex segmentation, a mixed-integer surface planner, Bezier-foot trajectories, and a 50 Hz whole-body MPC with a 400 Hz Riccati feedback controller. Key contributions include (i) a decoupled, high-frequency surface-selection framework, (ii) convex-plane-based perception with safety margins, (iii) an MPC framework that robustly handles contact slips and perception errors, demonstrated on challenging industrial stairs and parkour-like tasks, including ICRA 2023 simulations. The results show real-time performance, robust recovery from unplanned events, and applicability to diverse terrains, underscoring the practical impact for industrial robotics and autonomous inspection tasks. Overall, the work advances end-to-end perceptive locomotion by tightly integrating perception, planning, and whole-body control to push legged robots toward reliable operation in unstructured environments.

Abstract

Real-time synthesis of legged locomotion maneuvers in challenging industrial settings is still an open problem, requiring simultaneous determination of footsteps locations several steps ahead while generating whole-body motions close to the robot's limits. State estimation and perception errors impose the practical constraint of fast re-planning motions in a model predictive control (MPC) framework. We first observe that the computational limitation of perceptive locomotion pipelines lies in the combinatorics of contact surface selection. Re-planning contact locations on selected surfaces can be accomplished at MPC frequencies (50-100 Hz). Then, whole-body motion generation typically follows a reference trajectory for the robot base to facilitate convergence. We propose removing this constraint to robustly address unforeseen events such as contact slipping, by leveraging a state-of-the-art whole-body MPC (Croccodyl). Our contributions are integrated into a complete framework for perceptive locomotion, validated under diverse terrain conditions, and demonstrated in challenging trials that push the robot's actuation limits, as well as in the ICRA 2023 quadruped challenge simulation.

Figures (16)

• Figure 1: Industrial staircase descent with onboard perception. Video: https://youtu.be/bNVxTh0eixI.
• Figure 2: Overview of our perceptive locomotion pipeline. Around 1Hz, the perceptive elements Elevation map and Convex Plane Segmentation extract convex planes from the surrounding environment (Section \ref{['sec::perception']}). The Surface Selection block, running around 5Hz (depending on the gait chosen), chooses between them the next surfaces of contact (Section \ref{['sec::surface_selection']}). At the frequency of the MPC, 50Hz, Footstep and Collision free-trajectory elements generate the curve for each moving foot (Section \ref{['sec:foot_trajectory_generation']}). Finally, the Whole-Body MPC and Riccati gain controller synthesise the motion (Section \ref{['sec::motion_generation']}).
• Figure 3: Reduction to an 8-vertex polygon using the Visvalingam–Whyatt algorithm on an initial 20-vertex polygon on the left \ref{['fig::reduction_pts']}. Applying inner and outer margins to an 8-vertex polygon \ref{['fig::inner_outer']}.
• Figure 4: Example of surface processing and convex decomposition. These two figures represent the same 3D scene with 2 air overlapping the ground surface. On the right Fig. \ref{['fig::2D_decompo']}, the scene is viewed from a top perspective.
• Figure 5: (a) CoM extrapolation along the horizon. (b) ROMs of the 4 effectors for the current state. (c) ROMs along the horizon for the front right foot.