LVCP: LiDAR-Vision Tightly Coupled Collaborative Real-time Relative Positioning

TL;DR

This work tackles GPS-denied, map-free relative positioning between a monocular-camera drone and a LiDAR-equipped UGV by proposing LVCP, a tightly coupled LiDAR–vision framework. It introduces a coarse-to-fine pipeline that constructs point-plane associations and solves a Point-aided Bundle Adjustment to estimate the camera–LiDAR relative pose $T_{LC}$ and correct VIO drift, with an adaptive PSO-based sampling for real-time initialization. Key contributions include an initialization framework using multi-level sampling, a two-stage collaborative optimization, and a global pose-graph with loop closure, plus an extension to multi-agent scenarios where multiple drones localize within a shared LiDAR map. Experiments on EuRoC and dynamic, self-built datasets demonstrate real-time performance, robustness to initial pose disturbances, and accurate relative localization without prior maps, illustrating LVCP’s potential for coordinated air–ground missions in cluttered or GPS-denied environments.

Abstract

In air-ground collaboration scenarios without GPS and prior maps, the relative positioning of drones and unmanned ground vehicles (UGVs) has always been a challenge. For a drone equipped with monocular camera and an UGV equipped with LiDAR as an external sensor, we propose a robust and real-time relative pose estimation method (LVCP) based on the tight coupling of vision and LiDAR point cloud information, which does not require prior information such as maps or precise initial poses. Given that large-scale point clouds generated by 3D sensors has more accurate spatial geometric information than the feature point cloud generated by image, we utilize LiDAR point clouds to correct the drift in visual-inertial odometry (VIO) when the camera undergoes significant shaking or the IMU has a low signal-to-noise ratio. To achieve this, we propose a novel coarse-to-fine framework for LiDAR-vision collaborative localization. In this framework, we construct point-plane association based on spatial geometric information, and innovatively construct a point-aided Bundle Adjustment (BA) problem as the backend to simultaneously estimate the relative pose of the camera and LiDAR and correct the VIO drift. In this process, we propose a particle swarm optimization (PSO) based sampling algorithm to complete the coarse estimation of the current camera-LiDAR pose. In this process, the initial pose of the camera used for sampling is obtained based on VIO propagation, and the valid feature-plane association number (VFPN) is used to trigger PSO-sampling process. Additionally, we propose a method that combines Structure from Motion (SFM) and multi-level sampling to initialize the algorithm, addressing the challenge of lacking initial values.

Figures (17)

• Figure 1: The LiDAR and vision-based collaborative positioning system (LVCP) in a UGV and multi-UAVs scenario. An accurate point cloud map is constructed in real-time using LiDAR mounted on a UGV. Multiple small UAVs collect images and IMU data, synchronously registering the images to the dynamic point cloud while the UGV is in motion, thus achieving relative localization among multiple agents. Experimental results validate that the LVCP system can achieve accurate relative localization of UAVs without prior map and initial pose disturbances.
• Figure 2: Workflow of tightly coupled air-ground Localization positioning system. The process integrates multiple data sources, including IMU, image, and LiDAR data, and is organized into several key modules: pre-processing, initialization, optimization, and event-triggered sampling.
• Figure 3: An illustration of the notations and coordinate transformations.
• Figure 4: An illustration of the visual-inertial alignment and LiDAR-visual alignment processes for camera pose initialization.
• Figure 5: This figure illustrates the process of point cloud-aided camera initialization, divided into three moduals: Vision-only SFM, Vistual-Intertial Alignment and LiDAR-Vistual Alignment.