Occluded nuScenes: A Multi-Sensor Dataset for Evaluating Perception Robustness in Automated Driving

TL;DR

Occluded nuScenes provides the first multi-sensor, parameterized occlusion dataset built on nuScenes to enable reproducible evaluation of perception robustness under partial sensor failure. It offers four camera occlusions (dirt, water-blur, scratches, WoodScape soiling) with multiple severities and scripts for three degradation modes per radar and LiDAR, preserving compatibility with existing annotations. The dataset supports reproducible benchmarking of sensor fusion and resilience analysis, with pre-generated occluded camera data and on-demand radar/LiDAR occlusions, facilitating occlusion-aware training and robust perception in automated driving. This resource advances safety-critical perception research by enabling systematic, controlled studies of degradation effects across all primary sensing modalities.

Abstract

Robust perception in automated driving requires reliable performance under adverse conditions, where sensors may be affected by partial failures or environmental occlusions. Although existing autonomous driving datasets inherently contain sensor noise and environmental variability, very few enable controlled, parameterised, and reproducible degradations across multiple sensing modalities. This gap limits the ability to systematically evaluate how perception and fusion architectures perform under well-defined adverse conditions. To address this limitation, we introduce the Occluded nuScenes Dataset, a novel extension of the widely used nuScenes benchmark. For the camera modality, we release both the full and mini versions with four types of occlusions, two adapted from public implementations and two newly designed. For radar and LiDAR, we provide parameterised occlusion scripts that implement three types of degradations each, enabling flexible and repeatable generation of occluded data. This resource supports consistent, reproducible evaluation of perception models under partial sensor failures and environmental interference. By releasing the first multi-sensor occlusion dataset with controlled and reproducible degradations, we aim to advance research on robust sensor fusion, resilience analysis, and safety-critical perception in automated driving.

Figures (7)

• Figure 1: Overview of the camera occlusion generation pipeline. Multi-view images from the nuScenes dataset caesar2020nuscenes are processed to simulate four types of occlusion: (1) scratch effects created pngegg_scratch using overlay masks with background removal, (2) rain effects generated 10900335 through grid-based occlusion maps with randomised patch placement, scaling, and rotation, (3) dirt effects introduced 10900335 using opacity-weighted masks on high-impact regions, and (4) soiling patterns derived from WoodScape yogamani2019woodscape binary masks smoothed with a Gaussian filter. These occlusions mimic realistic visibility challenges for evaluating the robustness of perception.
• Figure 2: Overview of the Radar occlusion generation pipeline. Ground truth radar point clouds from five nuScenes sensors caesar2020nuscenes are altered using three occlusion types: (1) single sensor failure by removing data from selected radars, (2) weather-induced partial masking of returns, and (3) environmental disturbances via additive Gaussian noise.
• Figure 3: Overview of the LiDAR occlusion generation pipeline. Ground truth point clouds from the nuScenes dataset caesar2020nuscenes are modified using three occlusion types: (1) region-based removal of specific areas, (2) weather-induced partial masking, and (3) beam angle occlusion based on sensor direction.
• Figure 4: Examples of camera occlusion effects on nuScenes samples. The top row displays the original multi-view images, while the bottom rows illustrate the effects of dirt and rain applied at three increasing occlusion levels (0.1, 0.2, 0.3).
• Figure 5: