Know Where You're Uncertain When Planning with Multimodal Foundation Models: A Formal Framework

TL;DR

The paper addresses the problem of reliability gaps in robotic planning when using multimodal foundation models by disentangling two distinct uncertainty sources: perception uncertainty from visual understanding and decision uncertainty from plan generation. It combines conformal prediction for perception with Formal-Methods-Driven Prediction (FMDP) for decision uncertainty, enabling calibrated uncertainty quantification and formal guarantees on task satisfaction. Two interventions—active sensing to improve input quality and automated refinement to improve plan fidelity—are then applied, resulting in reduced output variability (up to 40%) and higher task-specification satisfaction (up to 5% gains) in both real and simulation environments. This framework supports robust, scalable autonomous planning and demonstrates strong Sim2Real transfer, with a clear pathway for extending uncertainty handling to additional sources and task descriptions.

Abstract

Multimodal foundation models offer a promising framework for robotic perception and planning by processing sensory inputs to generate actionable plans. However, addressing uncertainty in both perception (sensory interpretation) and decision-making (plan generation) remains a critical challenge for ensuring task reliability. We present a comprehensive framework to disentangle, quantify, and mitigate these two forms of uncertainty. We first introduce a framework for uncertainty disentanglement, isolating perception uncertainty arising from limitations in visual understanding and decision uncertainty relating to the robustness of generated plans. To quantify each type of uncertainty, we propose methods tailored to the unique properties of perception and decision-making: we use conformal prediction to calibrate perception uncertainty and introduce Formal-Methods-Driven Prediction (FMDP) to quantify decision uncertainty, leveraging formal verification techniques for theoretical guarantees. Building on this quantification, we implement two targeted intervention mechanisms: an active sensing process that dynamically re-observes high-uncertainty scenes to enhance visual input quality and an automated refinement procedure that fine-tunes the model on high-certainty data, improving its capability to meet task specifications. Empirical validation in real-world and simulated robotic tasks demonstrates that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines. These improvements are attributed to the combined effect of both interventions and highlight the importance of uncertainty disentanglement, which facilitates targeted interventions that enhance the robustness and reliability of autonomous systems. Fine-tuned models, code, and datasets are available at https://uncertainty-in-planning.github.io/.

Figures (12)

• Figure 1: Our planning framework disentangles perception and decision uncertainty, triggering the active sensing intervention. The framework improves the robustness of generated plans by reducing the propagation of perceptual inaccuracies.
• Figure 2: Our automated refinement framework generates high-certainty training data and fine-tunes the foundation model to improve its ability to generate plans that comply with task requirements.
• Figure 3: The first and second figures from left depict nonconformity distributions for perception and plan generation in simulated driving scenes (Carla) respectively.
• Figure 4: The first figure shows a scenario with high perception uncertainty due to the shadow and occlusion (perception uncertainty scores next to bounding boxes). The second figure shows a scenario with low perception uncertainty but high decision uncertainty, due to the inconsistency between the image and task description (traffic light is absent).
• Figure 5: Illustration of autonomous driving tasks wherein our strategy presented in Fig. \ref{['fig: strategy']} satisfies all task specifications during plan execution in both simulations and real-world environments.

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Definition 3
• proof