Dense 3D Displacement Estimation for Landslide Monitoring via Fusion of TLS Point Clouds and Embedded RGB Images

TL;DR

This work addresses the challenge of obtaining dense 3D displacement fields from TLS data in landslide monitoring by fusing densely sampled 3D geometry with co-registered RGB images. It introduces a three-level hierarchical partitioning scheme and a coarse-to-fine matching pipeline that combines 3D patch-based features and 2D image matches, followed by a refinement step and a Kabsch-ICP fine alignment to produce a dense DVF. The method achieves higher spatial coverage than the current state-of-the-art (F2S3) while maintaining comparable accuracy across two real landslide datasets, validated against external TS/GNSS observations and manual references. The approach is practical, scalable, and adaptable to other point clouds and monitoring tasks, with data and code publicly available for reproducibility and extension.

Abstract

Landslide monitoring is essential for understanding geohazards and mitigating associated risks. Existing point cloud-based methods, however, typically rely on either geometric or radiometric information and often yield sparse or non-3D displacement estimates. In this paper, we propose a hierarchical partitioning-based coarse-to-fine approach that integrates 3D point clouds and co-registered RGB images to estimate dense 3D displacement vector fields. Patch-level matches are constructed using both 3D geometry and 2D image features, refined via geometric consistency checks, and followed by rigid transformation estimation per match. Experimental results on two real-world landslide datasets demonstrate that the proposed method produces 3D displacement estimates with high spatial coverage (79% and 97%) and accuracy. Deviations in displacement magnitude with respect to external measurements (total station or GNSS observations) are 0.15 m and 0.25 m on the two datasets, respectively, and only 0.07 m and 0.20 m compared to manually derived references, all below the mean scan resolutions (0.08 m and 0.30 m). Compared with the state-of-the-art method F2S3, the proposed approach improves spatial coverage while maintaining comparable accuracy. The proposed approach offers a practical and adaptable solution for TLS-based landslide monitoring and is extensible to other types of point clouds and monitoring tasks. The example data and source code are publicly available at https://github.com/gseg-ethz/fusion4landslide.

Figures (19)

• Figure 1: Method overview. The proposed pipeline consists of five steps: (1) Point cloud tiles and the corresponding image tiles are generated. (2) Each paired source--target point cloud tile is partitioned into smaller patches. (3) Coarse matching establishes patch-wise correspondences / matches for each partition level based on both 3D point cloud and 2D RGB information. (4) A refinement module evaluates match quality and discards low-quality patch matches based on predefined criteria. (5) For each pair of patch matches, a rigid transformation is estimated and applied to all points within the source patch. The transformed source points and associated displacement vectors are then aggregated to construct 3D DVF, optionally producing point-to-point (P2P) matches. The final 3D DVF and P2P matches are integrated from different partition levels according to an integration step (cf.\ref{['sec:fine_matching']}).
• Figure 2: Hierarchical partitioning of the point cloud. Starting from the RGB-colored input, the method generates three levels of partitions that capture structural boundaries at multiple scales. The levels are generated independently using different regularization strengths: Level i (or level 1) preserves small, geometrically homogeneous parts; Level ii reveals larger structural components; and Level iii highlights entire objects or coherent regions, providing global contextual information robert2023spt.
• Figure 3: Different patch matching cases. Each case contains four example patch matches (same color) of point cloud $\mathbf{P}$ and $\mathbf{Q}$. Case (i): The patch match can differ in shape (see the rgb]0.24,0.53,0.58dark cyan ones). Case (ii): The patch match can move differently compared to the neighbor patches (see the different orientations of the rgb]1,0,0red arrows).
• Figure 4: Coarse matching based on patch feature similarity. (i) Network architecture for constructing patch features using point features extracted from a pretrained model within each patch. (ii) Mutual NN matching between patch features from the two epochs for generating candidate patch matches.
• Figure 5: Coarse matching based on statistic analysis.