Enhancing Monocular Height Estimation via Sparse LiDAR-Guided Correction

TL;DR

This work tackles the fragility of monocular height estimation under real-world illumination and data scarcity by introducing a fully automated correction pipeline that fuses dense MHE/MDE predictions with sparse ICESat-2 measurements. It provides a first comprehensive benchmark of seven calibration strategies, spanning Random Forest–based and parameter-efficient fine-tuning approaches, evaluated across six diverse regions. The method requires only a single georeferenced image and publicly available data, enabling globally scalable 3D height mapping. Results show substantial improvements in MAE and F1HE for both MHE and MDE backbones, with the optimal strategy depending on the backbone pretraining, demonstrating the practical viability of sparse LiDAR–guided correction for robust, high-resolution height mapping.

Abstract

Monocular height estimation (MHE) from very-high-resolution (VHR) optical imagery remains challenging due to limited structural cues and the high cost and geographic constraints of conventional elevation data such as airborne LiDAR and multi-view stereo. Although recent MHE and monocular depth estimation (MDE) models show strong performance, their robustness under varied illumination and scene conditions is still limited. We introduce a fully automated correction pipeline that integrates sparse, imperfect global LiDAR measurements from ICESat-2 with deep learning predictions to enhance accuracy and stability. The workflow relies entirely on publicly available models and data and requires only a single georeferenced optical image to produce corrected height maps, enabling low-cost and globally scalable deployment. We also establish the first benchmark for this task, evaluating two random forest based approaches, four parameter efficient fine tuning methods, and full fine tuning. Experiments across six diverse regions at 0.5 m resolution (297 km2), covering the urban cores of Tokyo, Paris, and Sao Paulo as well as suburban and forested areas, show substantial gains. The best method reduces the MHE model's mean absolute error (MAE) by 30.9 percent and improves its F1HE score by 44.2 percent. For the MDE model, MAE improves by 24.1 percent and the F1HE score by 25.1 percent. These results validate the effectiveness of our correction pipeline and demonstrate how sparse global LiDAR can systematically strengthen both MHE and MDE models, enabling scalable and widely accessible 3D height mapping.

Figures (12)

• Figure 1: Visualization of three lighting/texture conditions (top row) and the corresponding Grad-CAM attention maps with MAE values for two models (middle: MHE model RS3DAda, bottom: MDE model Depth Anything V2).
• Figure 2: Effect of varying shadow lengths on height estimation.
• Figure 3: Study areas with overlaid ICESat-2 tracks, illustrating land-cover composition, terrain elevation, building-height distributions, and the mean and standard deviation of above-ground object heights across selected urban, peri-urban, and forested regions.
• Figure 4: ICESat-2 satellite specifications.
• Figure 5: Overview of the proposed ICESat-2 based calibration pipeline. Raw photon data (ATL03/ATL08) are pre-processed and combined with MHE predictions directly, or with MDE predictions after a simple linear fitting against ICESat-2 tracks. Calibration is then performed using Random Forest or fine-tuning approaches, yielding a refined and spatially consistent height map.