Modeling and Measuring Redundancy in Multisource Multimodal Data for Autonomous Driving

TL;DR

Model and measure redundancy in multisource camera data and multimodal image-LiDAR data, and evaluate how removing redundant labels affects the YOLOv8 object detection task demonstrate that redundancy is a measurable and actionable DQ factor with direct implications for AV performance.

Abstract

Next-generation autonomous vehicles (AVs) rely on large volumes of multisource and multimodal ($M^2$) data to support real-time decision-making. In practice, data quality (DQ) varies across sources and modalities due to environmental conditions and sensor limitations, yet AV research has largely prioritized algorithm design over DQ analysis. This work focuses on redundancy as a fundamental but underexplored DQ issue in AV datasets. Using the nuScenes and Argoverse 2 (AV2) datasets, we model and measure redundancy in multisource camera data and multimodal image-LiDAR data, and evaluate how removing redundant labels affects the YOLOv8 object detection task. Experimental results show that selectively removing redundant multisource image object labels from cameras with shared fields of view improves detection. In nuScenes, mAP${50}$ gains from $0.66$ to $0.70$, $0.64$ to $0.67$, and from $0.53$ to $0.55$, on three representative overlap regions, while detection on other overlapping camera pairs remains at the baseline even under stronger pruning. In AV2, $4.1$-$8.6\%$ of labels are removed, and mAP${50}$ stays near the $0.64$ baseline. Multimodal analysis also reveals substantial redundancy between image and LiDAR data. These findings demonstrate that redundancy is a measurable and actionable DQ factor with direct implications for AV performance. This work highlights the role of redundancy as a data quality factor in AV perception and motivates a data-centric perspective for evaluating and improving AV datasets. Code, data, and implementation details are publicly available at: https://github.com/yhZHOU515/RedundancyAD

Figures (11)

• Figure 1: Illustration of multisource and multimodal data in autonomous vehicles. LiDAR point clouds, multi-camera images, RADAR, vehicle dynamics, user-related data, and external contextual inputs feed into the AV system. Redundancy arises from overlapping fields of view among multiple onboard cameras (green), as well as from multimodal sensing in which cameras and LiDAR (orange) observe the same objects. The rest colored blocks represent additional data sources and modalities.
• Figure 2: Illustration of research design, using nuScenes dataset as examples. We first quantify redundancy in multisource and multimodal autonomous driving datasets (RQ1), then design data pruning to remove redundant observations (RQ2), and finally evaluate how redundancy removal affects object detection performance (RQ3).
• Figure 3: The workflow of the research design on redundancy evaluation. nuScenes emphasizes 2D labeling and AV2 adopts 3D annotations. Beyond dataset-specific configurations, this workflow can be generalized to other AD datasets.
• Figure 4: Camera settings and overlapping FoVs in nuScenes. For datasets with different sensor layouts, the same redundancy modeling and measurement approach remains applicable by adjusting the geometric parameters.
• Figure 5: Illustration of cropping based on overlapping angles of the cameras.