Probabilistic Segmentation for Robust Field of View Estimation

TL;DR

This work addresses the vulnerability of field-of-view (FOV) estimation in autonomous vehicles to sensing attacks. It first shows that classical FOV estimators based on ray tracing and concave hulls are easily corrupted by LiDAR spoofing, then proposes a learning-based segmentation approach that uses BEV-projected LiDAR data and Monte Carlo dropout for uncertainty quantification. The model is trained on both benign and adversarial data and is augmented with an anomaly detector to flag unusual predictions, with extensive cross-dataset evaluation demonstrating strong generalization and robustness. Real-time integration in a ROS-based multi-agent setup confirms feasibility for deployment in autonomous systems, and open-sourced FOV datasets and code support reproducibility and further research.

Abstract

Attacks on sensing and perception threaten the safe deployment of autonomous vehicles (AVs). Security-aware sensor fusion helps mitigate threats but requires accurate field of view (FOV) estimation which has not been evaluated autonomy. To address this gap, we adapt classical computer graphics algorithms to develop the first autonomy-relevant FOV estimators and create the first datasets with ground truth FOV labels. Unfortunately, we find that these approaches are themselves highly vulnerable to attacks on sensing. To improve robustness of FOV estimation against attacks, we propose a learning-based segmentation model that captures FOV features, integrates Monte Carlo dropout (MCD) for uncertainty quantification, and performs anomaly detection on confidence maps. We illustrate through comprehensive evaluations attack resistance and strong generalization across environments. Architecture trade studies demonstrate the model is feasible for real-time deployment in multiple applications.

Figures (11)

• Figure 1: Space visible to the sensor is limited during occlusions from objects/infrastructure. Despite BEV LiDAR having a circularly symmetric FOV in wide open spaces, FOV is irregularly shaped in dense urban environments, and space behind nearby objects is not visible to the sensor. FOV algorithms fit model to visible space using point cloud as input and correctly captures lack of visibility behind objects.
• Figure 2: BEV of LiDAR point clouds from state of the art simulators and real-world datasets support the first labeled FOV dataset. Scenes are diverse and capture indoor and outdoor environments with varying levels of occlusion.
• Figure 3: The UGV maintains multiple sensors collecting time-synchronized data for perception and localization in dense, challenging indoor environments.
• Figure 4: FOV algorithms vulnerable to small number of spoof points - well under the demonstrated attacker capability from 2019cao-spoofing2020sun-spoofing. Adversarial training with UNet segmentation results in smoothed and robust FOV estimate.
• Figure 5: UNet architecture to ingest point cloud data mapped to BEV grid cells and output segmentation map. Utilizes a mixture of up and down convolutions with skip connections.