ME-CPT: Multi-Task Enhanced Cross-Temporal Point Transformer for Urban 3D Change Detection

TL;DR

This work tackles urban 3D semantic change detection from multi-temporal ALS point clouds, addressing cross-temporal feature alignment, class imbalance, and dataset scarcity. It introduces ME-CPT, a Multi-Task Enhanced Cross-Temporal Point Transformer that uses cross-temporal serialization and patch-level attention to learn semantic and change features jointly, aided by a semantic segmentation auxiliary task. A new NYC-SCD dataset (22.5 km$^2$) with four semantic classes and four change categories is released to benchmark performance. Empirical results across Urb3DCD-V2, SLPCCD, AHN-CD, and NYC-SCD demonstrate state-of-the-art performance, with ablations confirming the benefits of temporal indicators and multi-task supervision; the work also discusses transferability and future directions toward lighter models and broader change taxonomies.

Abstract

The point clouds collected by the Airborne Laser Scanning (ALS) system provide accurate 3D information of urban land covers. By utilizing multi-temporal ALS point clouds, semantic changes in urban area can be captured, demonstrating significant potential in urban planning, emergency management, and infrastructure maintenance. Existing 3D change detection methods struggle to efficiently extract multi-class semantic information and change features, still facing the following challenges: (1) the difficulty of accurately modeling cross-temporal point clouds spatial relationships for effective change feature extraction; (2) class imbalance of change samples which hinders distinguishability of semantic features; (3) the lack of real-world datasets for 3D semantic change detection. To resolve these challenges, we propose the Multi-task Enhanced Cross-temporal Point Transformer (ME-CPT) network. ME-CPT establishes spatiotemporal correspondences between point cloud across different epochs and employs attention mechanisms to jointly extract semantic change features, facilitating information exchange and change comparison. Additionally, we incorporate a semantic segmentation task and through the multi-task training strategy, further enhance the distinguishability of semantic features, reducing the impact of class imbalance in change types. Moreover, we release a 22.5 $km^2$ 3D semantic change detection dataset, offering diverse scenes for comprehensive evaluation. Experiments on multiple datasets show that the proposed MT-CPT achieves superior performance compared to existing state-of-the-art methods. The source code and dataset will be released upon acceptance at https://github.com/zhangluqi0209/ME-CPT.

Figures (13)

• Figure 1: Workflow of the proposed Multi-Task Enhanced Cross-Temporal Point Transformer (ME-CPT) for urban 3D semantic change detection.
• Figure 2: The difference between the proposed method and Siamese-based change detection networks in terms of neighborhood correspondence and feature extraction.
• Figure 3: The process of point cloud serialization for cross-temporal patches using different space-filling curves.
• Figure 4: Cross-temporal Attention Block.
• Figure 5: Multi-task enhanced feature extraction module.