Situationally-Aware Dynamics Learning

TL;DR

This work tackles robust autonomous navigation in unstructured environments by learning online representations of latent factors that influence robot dynamics. It frames the problem as a Generalized Hidden Parameter MDP and introduces a multivariate Bayesian Online Changepoint Detection to identify different underlying data-generating processes, mapping them to symbolic situation representations that condition a dynamics model. An ensemble of Gaussian neural networks, augmented with situation symbols, forms a situationally-aware dynamics model that is planned with Model Predictive Path Integral (MPPI), achieving faster learning, improved data efficiency, and safer, adaptive navigation in both simulation and real-world terrains. The approach demonstrates strong generalization to unseen terrains, data-efficient online adaptation without privileged information, and emergent safe behaviors, underscoring its potential for real-time, context-aware robotic decision-making.

Abstract

Autonomous robots operating in complex, unstructured environments face significant challenges due to latent, unobserved factors that obscure their understanding of both their internal state and the external world. Addressing this challenge would enable robots to develop a more profound grasp of their operational context. To tackle this, we propose a novel framework for online learning of hidden state representations, with which the robots can adapt in real-time to uncertain and dynamic conditions that would otherwise be ambiguous and result in suboptimal or erroneous behaviors. Our approach is formalized as a Generalized Hidden Parameter Markov Decision Process, which explicitly models the influence of unobserved parameters on both transition dynamics and reward structures. Our core innovation lies in learning online the joint distribution of state transitions, which serves as an expressive representation of latent ego- and environmental-factors. This probabilistic approach supports the identification and adaptation to different operational situations, improving robustness and safety. Through a multivariate extension of Bayesian Online Changepoint Detection, our method segments changes in the underlying data generating process governing the robot's dynamics. The robot's transition model is then informed with a symbolic representation of the current situation derived from the joint distribution of latest state transitions, enabling adaptive and context-aware decision-making. To showcase the real-world effectiveness, we validate our approach in the challenging task of unstructured terrain navigation, where unmodeled and unmeasured terrain characteristics can significantly impact the robot's motion. Extensive experiments in both simulation and real world reveal significant improvements in data efficiency, policy performance, and the emergence of safer, adaptive navigation strategies.

Figures (19)

• Figure 1: Motivating Example. When the robot has access only to incomplete state information, the same action, $\mathbf{a}$, can yield significantly different outcomes. In this example, unmodeled terrain types—factors absent from the state and transition model—affect the robot's response, resulting in distinct changes in position and heading ($\Delta x$, $\Delta y$, and $\Delta \psi$, respectively) for the same control action (commanded linear and angular velocity). Generalized Hidden Parameter Markov Decision Processes (GHP-MDPs) address this challenge by modeling the unknown latent factors responsible for these discrepancies and incorporating them into the transition model, thereby enhancing predictive accuracy. (Illustrative environment images generated with Gemini AI.)
• Figure 2: System overview. Our situation identification method leverages the observable state to model online the local transition distribution, in order to build a representation of how the latent factors currently affect the robot's dynamics. In other words, the learned representation encompasses the hidden or unobservable component of the state. Using the recovered information, the model-based planner can consider the particularities of the current situation via an informed transition model.
• Figure 3: Hardware setup used during real-world experiments.
• Figure 4: Some real-world scenarios used to evaluate and compare the proposed dynamics model against the competitive baselines. (Left) Navigation across diverse terrain types, including mulch, grass, concrete, and mud, each with distinct surface properties and geometries. The final section involves a short but steep climb on mulch and grass, which presents a significant challenge due to reduced traction. (Right) Traversing a highly demanding terrain with steep ascents and descents, compounded by numerous bumps and irregularities. These features make the environment difficult to navigate safely and efficiently. Notably, our method is the only one capable of successfully completing this scenario.
• Figure 5: Training task in the Inspection World Environmentinspection_world. The region around waypoint #3 is slippery and difficult to reach.

Theorems & Definitions (7)

• Definition 1
• Lemma 1: Growth and Changepoint Probabilities
• proof
• Lemma 2: Online Distribution Parameters Learning
• proof
• Lemma 3: Observation Posterior
• proof