UF-RNN: Real-Time Adaptive Motion Generation Using Uncertainty-Driven Foresight Prediction

TL;DR

UF-RNN addresses robust robotic behavior under state uncertainty by augmenting a predictive RNN with a Foresight module that internally simulates multiple futures and selects actions that minimize future uncertainty. The architecture combines a Convolutional Autoencoder with a Stochastic Hierarchical LSTM and a closed-loop foresight mechanism, guided by multiple noise perturbations and adaptive noise intensity. Across simulated and real-world door-opening tasks, UF-RNN demonstrates earlier learning, higher success rates, and interpretable chaotic latent dynamics that enable targeted exploration at uncertainty-triggering moments. This approach shows that uncertainty-driven foresight can significantly enhance exploration efficiency and policy robustness in uncertain, real-world robotic environments.

Abstract

Training robots to operate effectively in environments with uncertain states, such as ambiguous object properties or unpredictable interactions, remains a longstanding challenge in robotics. Imitation learning methods typically rely on successful examples and often neglect failure scenarios where uncertainty is most pronounced. To address this limitation, we propose the Uncertainty-driven Foresight Recurrent Neural Network (UF-RNN), a model that combines standard time-series prediction with an active "Foresight" module. This module performs internal simulations of multiple future trajectories and refines the hidden state to minimize predicted variance, enabling the model to selectively explore actions under high uncertainty. We evaluate UF-RNN on a door-opening task in both simulation and a real-robot setting, demonstrating that, despite the absence of explicit failure demonstrations, the model exhibits robust adaptation by leveraging self-induced chaotic dynamics in its latent space. When guided by the Foresight module, these chaotic properties stimulate exploratory behaviors precisely when the environment is ambiguous, yielding improved success rates compared to conventional stochastic RNN baselines. These findings suggest that integrating uncertainty-driven foresight into imitation learning pipelines can significantly enhance a robot's ability to handle unpredictable real-world conditions.

Figures (6)

• Figure 1: Example of a task that embrace high uncertainty. Opening doors may require the robot to interact before understanding which way the door can be opened, especially when it is not visually distinguishable.
• Figure 2: Network structure of the proposed UF-RNN model. (A) The model consists of an RNN module and a novel Foresight Module, which is used to refine the RNN hidden states through internal simulations. (B) The structure of RNN module, which uses a stochastic hierarchical RNN (SH-RNN) for predicting the expected sensor values at the next time step. (C) describes the process of Foresight Module which re-directs the RNN hidden states to guide to less uncertain states via internal simulation (closed-loop prediction).
• Figure 3: Comparison of success rates per training model. The graph shows the transition of success rates using the trained model every 100 epochs, up to 3000. The graph is a stacked plot color coded by door types, showing the success rates of each type per 10 trials.
• Figure 4: Comparison on transitions of RNN hidden states per training model type. The graph plots the hidden states of $LSTM_{shared}$, which is compressed to two dimensions using principal component analysis. The dotted lines show prediction of three motions on pre-collected data (offline), and the solid line shows the prediction on pushing motion during inference (online).
• Figure 5: Transition of Lyapunov exponents compared between training model types. Highlighted area display distinct peaks of the exponent for each model, where higher values indicate that highly chaotic properties are structured at the respective timestep. The structurization of chaotic properties also display at what timing the model considered the environment to have high uncertainty.