Synthesizing and Identifying Noise Levels in Autonomous Vehicle Camera Radar Datasets
Mathis Morales, Golnaz Habibi
TL;DR
This work tackles robustness of autonomous-vehicle perception under sensor faults by introducing a realistic camera–radar data synthesis pipeline and a lightweight noise-recognition system. The authors implement dual noise-level dials ($N_{lvl}$) for camera and radar and train separate lightweight models (camera: U‑Net; radar: shallow conv nets) to predict 11 degradation levels, enabling downstream detectors to adapt to degraded data. Camera degradation comprises blur, exposure changes, and additive noise, while radar degradation adds ghost points, RCS-based false negatives, and noise-induced range/velocity shifts, all governed by relations involving $SNR$ such as $SNR'_dB = SNR_dB - N/10$ and $SNR'_lin = 10^{SNR'_dB/10}$. On nuScenes-based data, the approach yields an overall accuracy of about $54.4 ext{%}$ across 11 classes (camera $62.2 ext{%}$, radar $20.8 ext{%}$), demonstrating a detector-agnostic pathway to quantify and mitigate degradation in multi-sensor perception. This framework lays groundwork for robust 3D perception under realistic sensor interference and can be extended to adverse weather and radar jamming scenarios, with potential improvements in model efficiency and coverage of more failure modes.
Abstract
Detecting and tracking objects is a crucial component of any autonomous navigation method. For the past decades, object detection has yielded promising results using neural networks on various datasets. While many methods focus on performance metrics, few projects focus on improving the robustness of these detection and tracking pipelines, notably to sensor failures. In this paper we attempt to address this issue by creating a realistic synthetic data augmentation pipeline for camera-radar Autonomous Vehicle (AV) datasets. Our goal is to accurately simulate sensor failures and data deterioration due to real-world interferences. We also present our results of a baseline lightweight Noise Recognition neural network trained and tested on our augmented dataset, reaching an overall recognition accuracy of 54.4\% on 11 categories across 10086 images and 2145 radar point-clouds.