Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation

TL;DR

This work tackles safe, platform-aware robotic navigation in rough terrain by learning a perceptive forward dynamics model that conditions future state predictions on surrounding geometry and proprioceptive history. The model is trained with a hybrid mix of synthetic and real data to capture full system dynamics and is integrated into a zero-shot MPPI planning framework, enabling simple reward formulations while maintaining safety. Key contributions include the first application of a rough-terrain forward dynamics model to a quadruped in sim-to-real transfer, a hybrid data collection/training strategy, and a simplified planning cost that leverages learned risk predictions. The approach shows substantial gains on the ANYmal platform, with improved position estimation and higher navigation success in rough environments, and the authors provide public code for reproducibility.

Abstract

Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot`s capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable. To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot`s future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers, and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation by on average 41% over competitive baselines, which translates into a 27% higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data. Code and models are made publicly available under https://github.com/leggedrobotics/fdm.

Figures (4)

• Figure 1: Demonstration of the proposed perceptive Forward Dynamics Model for robust navigation in complex environments. The model, trained with real-world and simulation data, predicts the robot’s future states given a sequence of velocity actions. It takes as input the surrounding geometry in the form of a height scan, along with past states and proprioceptive measurements. A sampling-based planner evaluates the integrated paths based on simple reward functions to select the optimal next action in a receding horizon fashion. (A) Ten example paths are visualized and overlaid on the environment image alongside the height map and the downsampled height scan (blue points). Path colors indicate rewards, with the highest reward assigned to the closest collision-free trajectory to the goal. (B–E) Additional planning events are shown, displaying sampled paths and the selected trajectory (green), demonstrating safe planning in rough terrain.
• Figure 2: Overview of the FDM training. Data is collected in a parallelized simulation setting and from real-world experiments. The proprioceptive and exteroceptive measurements, along with velocity actions, are saved in a replay buffer from which training data is sampled. The information about the current and past state of the robotic system is encoded and given to a recurrent unit, which generates a latent of the robot's future states conditioned on the applied actions. Different heads are used to predict the future SE2 poses and failure probabilities.
• Figure 3: Overview of the MPPI-based planning approach. A population of action trajectories is generated by perturbating an initial solution with Gaussian noise. The presented FDM is then used to predict the future states and the risk of the individual action sequences, which are evaluated using a reward formulation. After $k$ iterations, using the previous highest reward action sequence as a starting point, the sequence with the maximum reward is executed.
• Figure :