Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Robustness of LiDAR-Camera Fusion Model against Weather Corruption from Fusion Strategy Perspective

Yihao Huang, Kaiyuan Yu, Qing Guo, Felix Juefei-Xu, Xiaojun Jia, Tianlin Li, Geguang Pu, Yang Liu

TL;DR

This work probes the robustness of LiDAR–camera fusion for 3D object detection under weather corruption, analyzing robustness through the lens of fusion strategies. It identifies that fusion modality greatly shapes resilience, with virtual point-based fusion delivering the strongest robustness across weather scenarios. The authors propose a lightweight, general method to improve robustness by dynamically weighting LiDAR and camera features during fusion, using both Sigmoid-based and cross-attention-based implementations, and validate gains across eight fusion variants on a synthesized KITTI-C dataset. Results show consistent performance gains on corrupted data while preserving or modestly improving clean-data performance, underscoring the practical value of adaptive fusion weighting for real-world autonomy under adverse weather.

Abstract

In recent years, LiDAR-camera fusion models have markedly advanced 3D object detection tasks in autonomous driving. However, their robustness against common weather corruption such as fog, rain, snow, and sunlight in the intricate physical world remains underexplored. In this paper, we evaluate the robustness of fusion models from the perspective of fusion strategies on the corrupted dataset. Based on the evaluation, we further propose a concise yet practical fusion strategy to enhance the robustness of the fusion models, namely flexibly weighted fusing features from LiDAR and camera sources to adapt to varying weather scenarios. Experiments conducted on four types of fusion models, each with two distinct lightweight implementations, confirm the broad applicability and effectiveness of the approach.

Improving Robustness of LiDAR-Camera Fusion Model against Weather Corruption from Fusion Strategy Perspective

TL;DR

This work probes the robustness of LiDAR–camera fusion for 3D object detection under weather corruption, analyzing robustness through the lens of fusion strategies. It identifies that fusion modality greatly shapes resilience, with virtual point-based fusion delivering the strongest robustness across weather scenarios. The authors propose a lightweight, general method to improve robustness by dynamically weighting LiDAR and camera features during fusion, using both Sigmoid-based and cross-attention-based implementations, and validate gains across eight fusion variants on a synthesized KITTI-C dataset. Results show consistent performance gains on corrupted data while preserving or modestly improving clean-data performance, underscoring the practical value of adaptive fusion weighting for real-world autonomy under adverse weather.

Abstract

In recent years, LiDAR-camera fusion models have markedly advanced 3D object detection tasks in autonomous driving. However, their robustness against common weather corruption such as fog, rain, snow, and sunlight in the intricate physical world remains underexplored. In this paper, we evaluate the robustness of fusion models from the perspective of fusion strategies on the corrupted dataset. Based on the evaluation, we further propose a concise yet practical fusion strategy to enhance the robustness of the fusion models, namely flexibly weighted fusing features from LiDAR and camera sources to adapt to varying weather scenarios. Experiments conducted on four types of fusion models, each with two distinct lightweight implementations, confirm the broad applicability and effectiveness of the approach.
Paper Structure (19 sections, 1 equation, 10 figures, 16 tables)

This paper contains 19 sections, 1 equation, 10 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Introduction
  3. Related Work
  4. LiDAR-camera Fusion Detection
  5. Robustness of LiDAR-camera Fusion Methods against Weather Corruption
  6. Evaluation
  7. Weather corruption, Datasets, and Metrics
  8. Type of LiDAR-camera Fusion Models
  9. Evaluation Results against Weather Corruption
  10. Influence of Corrupted Single Modal
  11. Visualization Analysis across Different Weather
  12. Enhancement Method and Experiment
  13. Enhancement Method
  14. Enhancement Results
  15. Weather Corruption Details
  16. ...and 4 more sections

Figures (10)

  • Figure 1: Flexible weighted fusion between LiDAR and camera branches of the model improves the robustness of the LiDAR-camera fusion model against weather corruption.
  • Figure 2: Points cloud and image under weather corruption.
  • Figure 3: RCE results of SOTA LiDAR-camera fusion models across four fusion modals on KITTI-C dataset.
  • Figure 4: RCE results of fair LiDAR-camera fusion models across four fusion modals on KITTI-C dataset.
  • Figure 5: Comparison between clean and corrupted LiDAR/camera information respectively across four high-level weather corruption. Red points are from clean information while blue points are from weather corruption.
  • ...and 5 more figures