Learning to Navigate Under Imperfect Perception: Conformalised Segmentation for Safe Reinforcement Learning

TL;DR

This work addresses safe autonomous navigation under imperfect perception by introducing COPPOL, a framework that embeds conformal prediction into hazard segmentation to produce set-valued hazard maps with finite-sample safety guarantees. These calibrated maps are then used to construct a risk-aware cost field for downstream reinforcement learning-based path planning, bridging perception and control in a principled way. Empirically, COPPOL achieves substantially higher hazard coverage (4–6x) and safer trajectories without sacrificing efficiency, and demonstrates robustness to distributional shift through noisy observations. The approach is modular and agnostic to the conformal predictor used, offering a practical path to reliable navigation in safety-critical environments.

Abstract

Reliable navigation in safety-critical environments requires both accurate hazard perception and principled uncertainty handling to strengthen downstream safety handling. Despite the effectiveness of existing approaches, they assume perfect hazard detection capabilities, while uncertainty-aware perception approaches lack finite-sample guarantees. We present COPPOL, a conformal-driven perception-to-policy learning approach that integrates distribution-free, finite-sample safety guarantees into semantic segmentation, yielding calibrated hazard maps with rigorous bounds for missed detections. These maps induce risk-aware cost fields for downstream RL planning. Across two satellite-derived benchmarks, COPPOL increases hazard coverage (up to 6x) compared to comparative baselines, achieving near-complete detection of unsafe regions while reducing hazardous violations during navigation (up to approx 50%). More importantly, our approach remains robust to distributional shift, preserving both safety and efficiency.

Figures (6)

• Figure 1: High-level overview of COPPOL, our conformal-driven perception-to-policy learning approach for safe navigation over risky terrain. COPPOL combines (A) real-world image capture via satellites of the rover's surrounding terrain, (B) U-Net-based segmentation of hazards with uncertainty calibration through conformal prediction, and (C) an RL agent to plan a safe trajectory to the goal. Orange modules represent the environment; blue modules are learned components of our approach.
• Figure 2: Comparison of hazard predictions for (a) DeepMoon and (b) YTU-WaterNet hazards. Each row shows the input image captured via satellite, ground truth mask, and the output of different methods: Baseline U-Net, MC, CRC, and MC-CP.
• Figure 3: Performance of comparative methods on four DeepMoon environments (vector view). Curves show average episodic reward, success rate, distance to ground truth hazard, and time spent in ground-truth hazards for Baseline, MC, CRC, and MC-CP (shaded bands indicate variability).
• Figure 4: Performance of comparative methods on four YTU-WaterNet environments (vector view). Curves show average episodic reward, success rate, distance to ground truth hazard, and time spent in ground-truth hazards for Baseline, MC, CRC, and MC-CP (shaded bands indicate variability).
• Figure 5: Performance of comparative methods on Environment 1 from the DeepMoon environments using the whole predictive hazard map as the state observation (image view). Curves show average episodic reward, success rate, distance to ground truth hazard, and time spent in ground-truth hazards for Baseline, MC, CRC, and MC-CP (shaded bands indicate variability).