Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safe Planning in Unknown Environments Using Conformalized Semantic Maps

David Smith Sundarsingh, Yifei Li, Tianji Tang, George J. Pappas, Nikolay Atanasov, Yiannis Kantaros

TL;DR

This paper presents the first planner for semantic reach-avoid tasks that achieves user-specified mission completion rates without requiring any knowledge of sensor models or noise, enabled by quantifying uncertainty in semantic maps, constructed on-the-fly from perceptual measurements, using conformal prediction in a model and distribution free manner.

Abstract

This paper addresses semantic planning problems in unknown environments under perceptual uncertainty. The environment contains multiple unknown semantically labeled regions or objects, and the robot must reach desired locations while maintaining class-dependent distances from them. We aim to compute robot paths that complete such semantic reach-avoid tasks with user-defined probability despite uncertain perception. Existing planning algorithms either ignore perceptual uncertainty, thus lacking correctness guarantees, or assume known sensor models and noise characteristics. In contrast, we present the first planner for semantic reach-avoid tasks that achieves user-specified mission completion rates without requiring any knowledge of sensor models or noise. This is enabled by quantifying uncertainty in semantic maps, constructed on-the-fly from perceptual measurements, using conformal prediction in a model and distribution free manner. We validate our approach and the theoretical mission completion rates through extensive experiments, showing that it consistently outperforms baselines in mission success rates.

Safe Planning in Unknown Environments Using Conformalized Semantic Maps

TL;DR

This paper presents the first planner for semantic reach-avoid tasks that achieves user-specified mission completion rates without requiring any knowledge of sensor models or noise, enabled by quantifying uncertainty in semantic maps, constructed on-the-fly from perceptual measurements, using conformal prediction in a model and distribution free manner.

Abstract

This paper addresses semantic planning problems in unknown environments under perceptual uncertainty. The environment contains multiple unknown semantically labeled regions or objects, and the robot must reach desired locations while maintaining class-dependent distances from them. We aim to compute robot paths that complete such semantic reach-avoid tasks with user-defined probability despite uncertain perception. Existing planning algorithms either ignore perceptual uncertainty, thus lacking correctness guarantees, or assume known sensor models and noise characteristics. In contrast, we present the first planner for semantic reach-avoid tasks that achieves user-specified mission completion rates without requiring any knowledge of sensor models or noise. This is enabled by quantifying uncertainty in semantic maps, constructed on-the-fly from perceptual measurements, using conformal prediction in a model and distribution free manner. We validate our approach and the theoretical mission completion rates through extensive experiments, showing that it consistently outperforms baselines in mission success rates.

Paper Structure

This paper contains 18 sections, 2 theorems, 10 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Robot & Environment Modeling
  4. Reachability Task with Semantic Safety Constraints
  5. Perception System and Semantic Mapping
  6. Problem Statement
  7. Safe Semantic Planning in Unknown Environments
  8. Conformalizing Semantic Maps
  9. Planning over Conformalized Semantic Maps
  10. Performance Guarantees
  11. Experimental Validation
  12. Setting Up Experiments and the Proposed Planner
  13. Empirical Evaluation of the Proposed Planner
  14. Effect of Number $|\hat{{\mathcal{P}}}|$ of Calibration Plans
  15. Comparative Experiments
  16. ...and 3 more sections

Key Result

Proposition 4.1

Consider a test environment $\Omega_{\text{test}}\sim{\mathcal{D}}$ with unknown true labels $\{k_j^{\text{gt}}\}_{j\in{\mathcal{J}}}$. Given any path ${\mathbf x}_{0:H}$ in $\Omega_{\text{test}}$ yielding a sequence of maps ${\mathbf m}_{0:H}$, the prediction sets $\{{\mathcal{C}}_j({\mathbf m}_t)\

Figures (5)

  • Figure 1: Illustration of the proposed planning framework for semantic reach-avoid tasks $\psi$. Given categorical and range measurements, a semantic map is constructed ssmi_temp. To account for map uncertainty caused by perceptual imperfections, conformal prediction is employed to generate sets of maps that contain the ground-truth environment with a user-defined probability. These sets are then used by a planner to generate uncertainty-aware paths that complete the assigned tasks with a user-defined probability $1-\alpha$.
  • Figure 2: The mapping algorithm uses depth and categorical measurements ${\mathcal{Z}}_t$, provided as a depth image and a segmented image, respectively, to generate a 3D semantic grid map ${\mathbf m}_t$. Each grid cell is associated with a semantic category, and its color corresponds to that category.
  • Figure 3: Illustration of a path ${\mathbf x}_{0:H}$ to the goal region (green disk) consisting of an exploration sub-path (red) and two exploitation sub-paths (green). The red regions model the unsafe regions as per \ref{['equ:pathSatisfy']}.
  • Figure 4: Top view of the environment (left) and the corresponding map ${\mathbf m}_t$ (right). The most likely label in ${\mathbf m}_t$ for cell $j$, occupied by a human, is 'tree’. In contrast, the prediction set constructed by our method for that cell mitigates this mapping error by including 'tree’, 'human’, and 'free-space’.
  • Figure 5: Hardware demonstration of the proposed method.

Theorems & Definitions (5)

  • Proposition 4.1: Mapping Guarantees
  • proof
  • Theorem 4.3: Planning Guarantees
  • proof
  • Remark 4.4: Dataset-conditional guarantees