Cooperative Visual-LiDAR Extrinsic Calibration Technology for Intersection Vehicle-Infrastructure: A review

TL;DR

This survey addresses cooperative extrinsic calibration for camera-LiDAR systems in intersection environments, spanning vehicle-side, roadside, and vehicle-road collaboration. It establishes the mathematical foundations of intrinsic and extrinsic calibration, and analyzes single-end and multi-end calibration approaches, including target-based and targetless methods, explicit and implicit dynamic strategies. The work highlights the state of the art in static infrastructure calibration, dynamic joint calibration, and data-driven approaches, while discussing datasets (simulated and real) and application contexts for cooperative perception and positioning. Its contributions lie in organizing a comprehensive view of multi-end calibration challenges, methods, and future directions, with practical implications for robust, scalable autonomous driving at complex urban intersections.

Abstract

In the typical urban intersection scenario, both vehicles and infrastructures are equipped with visual and LiDAR sensors. By successfully integrating the data from vehicle-side and road monitoring devices, a more comprehensive and accurate environmental perception and information acquisition can be achieved. The Calibration of sensors, as an essential component of autonomous driving technology, has consistently drawn significant attention. Particularly in scenarios involving multiple sensors collaboratively perceiving and addressing localization challenges, the requirement for inter-sensor calibration becomes crucial. Recent years have witnessed the emergence of the concept of multi-end cooperation, where infrastructure captures and transmits surrounding environment information to vehicles, bolstering their perception capabilities while mitigating costs. However, this also poses technical complexities, underscoring the pressing need for diverse end calibration. Camera and LiDAR, the bedrock sensors in autonomous driving, exhibit expansive applicability. This paper comprehensively examines and analyzes the calibration of multi-end camera-LiDAR setups from vehicle, roadside, and vehicle-road cooperation perspectives, outlining their relevant applications and profound significance. Concluding with a summary, we present our future-oriented ideas and hypotheses.

Figures (4)

• Figure 1: Schematic, intelligent connected cars and road infrastructure collaboration on a traffic scenario.
• Figure 2: Schematic, intelligent connected cars and road infrastructure collaboration on a traffic scenario.Infra is short for road infrastructure.
• Figure 3: Collaborative Visual-LiDAR extrinsic calibration process: (1)Data Collection for Calibration Objects: Camera and LiDAR data are collected simultaneously at the vehicle and infrastructure ends. Prior to this, the camera needs to undergo intrinsic calibration. (2)Individual Camera and LiDAR extrinsic calibration are separately completed at the vehicle and infrastructure ends to ensure consistent representation of camera and LiDAR data on the same end, thereby enabling single-end visual-LiDAR data fusion. The specific method varies based on the need for specific artificial markers, categorized into requiring target calibration and targetless calibration.(3) Collaborative extrinsic calibration is performed at the vehicle and road ends. Depending on whether specific extrinsic are directly output, methods are categorized into implicit and explicit methods. (4) The fused Visual-LiDAR data obtained from calibration can be used for various tasks after different decoding and post-processing, using different information in diverse applications.
• Figure 4: Illustration of Visual-LiDAR Calibration Using a 2D Checkerboard Calibration Board.