Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey

TL;DR

This work elaborates on the fundamental principles, methodologies, and applications of radar-camera fusion perception, and dives into the key techniques concerning sensor calibration, modal representation, data alignment, and fusion operation.

Abstract

Multi-modal fusion is imperative to the implementation of reliable object detection and tracking in complex environments. Exploiting the synergy of heterogeneous modal information endows perception systems the ability to achieve more comprehensive, robust, and accurate performance. As a nucleus concern in wireless-vision collaboration, radar-camera fusion has prompted prospective research directions owing to its extensive applicability, complementarity, and compatibility. Nonetheless, there still lacks a systematic survey specifically focusing on deep fusion of radar and camera for object detection and tracking. To fill this void, we embark on an endeavor to comprehensively review radar-camera fusion in a holistic way. First, we elaborate on the fundamental principles, methodologies, and applications of radar-camera fusion perception. Next, we delve into the key techniques concerning sensor calibration, modal representation, data alignment, and fusion operation. Furthermore, we provide a detailed taxonomy covering the research topics related to object detection and tracking in the context of radar and camera technologies.Finally, we discuss the emerging perspectives in the field of radar-camera fusion perception and highlight the potential areas for future research.

Figures (22)

• Figure 1: Sensor characteristics of radar, camera and LiDAR.
• Figure 2: The organization and taxonomy of the article.
• Figure 3: An exemplary application of multimodal fusion perception in autonomous driving.
• Figure 4: Major elements in the radar working pipeline.
• Figure 5: A thorough overview of public datasets with radar and camera data for object detection and tracking.