A Multi-Sensor Fusion Approach for Rapid Orthoimage Generation in Large-Scale UAV Mapping

TL;DR

This work tackles the challenge of rapidly generating large-scale, georeferenced orthoimages from UAV data by integrating a multi-sensor UAV platform (GPS, IMU, 4D millimeter-wave radar, and camera). It introduces a prior-pose informed feature matching strategy and a radar-derived terrain map to accelerate Structure-from-Motion and produce robust orthoimages in complex terrains, particularly farmland. Key contributions include a multi-sensor synchronization framework using an EKF, radar-based terrain maps as DTMs, and a fast, block-based feature matching method that reduces the number of features needed for accurate reconstruction. Experimental results show substantial speedups in feature matching (up to 25× for brute-force and 5× for FLANN) and near-parity in geospatial alignment with ground truth, along with significant timing advantages over commercial pipelines like WebODM and Photoscan. The approach demonstrates practical impact for agricultural monitoring and large-area mapping where robustness and speed are critical.

Abstract

Rapid generation of large-scale orthoimages from Unmanned Aerial Vehicles (UAVs) has been a long-standing focus of research in the field of aerial mapping. A multi-sensor UAV system, integrating the Global Positioning System (GPS), Inertial Measurement Unit (IMU), 4D millimeter-wave radar and camera, can provide an effective solution to this problem. In this paper, we utilize multi-sensor data to overcome the limitations of conventional orthoimage generation methods in terms of temporal performance, system robustness, and geographic reference accuracy. A prior-pose-optimized feature matching method is introduced to enhance matching speed and accuracy, reducing the number of required features and providing precise references for the Structure from Motion (SfM) process. The proposed method exhibits robustness in low-texture scenes like farmlands, where feature matching is difficult. Experiments show that our approach achieves accurate feature matching orthoimage generation in a short time. The proposed drone system effectively aids in farmland detection and management.

Figures (7)

• Figure 1: Rapid orthoimage generation pipeline. Consecutive frames undergo feature extraction and matching using 4D radar terrain height and GPS-IMU pose data. Poses are corrected via SfM, and a simplified DTM is generated from the radar point cloud. The orthoimage is then produced by combining corrected poses and the DTM.
• Figure 2: Schematic of the prior-pose-optimized feature matching method. Scene height $\mathbf{Z_{W}}$ from radar, along with camera parameters and prior pose, is used to back-project feature points from image A into 3D coordinates and then project them into image B's pixel coordinate system. The pixel plane is divided into blocks for matching, with the search area enlarged due to initial pose uncertainty.
• Figure 3: Trajectories evaluation with evo
• Figure 4: Qualitative comparison before and after optimization on both scenes
• Figure 5: Simplified terrain map for orthoimages