STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset

TL;DR

STPLS3D tackles the difficulty of producing large-scale, richly annotated 3D aerial data by introducing a fully automatic synthetic data pipeline that mirrors real UAV photogrammetry and combines it with real-world data. The pipeline leverages GIS-based environment generation, photorealistic 2D rendering, and automatic semantic/instance annotation to create a 16+ km^2 synthetic landscape with 18 semantic and 14 instance classes, plus a real-world subset for validation. Experiments show that incorporating synthetic data improves 3D semantic and generalization performance on real data, especially when combined with real data, though small-object and domain-gap issues persist. The work highlights synthetic data as a practical avenue for scalable pretraining, domain adaptation, and broader 3D vision tasks in outdoor environments.

Abstract

Although various 3D datasets with different functions and scales have been proposed recently, it remains challenging for individuals to complete the whole pipeline of large-scale data collection, sanitization, and annotation. Moreover, the created datasets usually suffer from extremely imbalanced class distribution or partial low-quality data samples. Motivated by this, we explore the procedurally synthetic 3D data generation paradigm to equip individuals with the full capability of creating large-scale annotated photogrammetry point clouds. Specifically, we introduce a synthetic aerial photogrammetry point clouds generation pipeline that takes full advantage of open geospatial data sources and off-the-shelf commercial packages. Unlike generating synthetic data in virtual games, where the simulated data usually have limited gaming environments created by artists, the proposed pipeline simulates the reconstruction process of the real environment by following the same UAV flight pattern on different synthetic terrain shapes and building densities, which ensure similar quality, noise pattern, and diversity with real data. In addition, the precise semantic and instance annotations can be generated fully automatically, avoiding the expensive and time-consuming manual annotation. Based on the proposed pipeline, we present a richly-annotated synthetic 3D aerial photogrammetry point cloud dataset, termed STPLS3D, with more than 16 $km^2$ of landscapes and up to 18 fine-grained semantic categories. For verification purposes, we also provide a parallel dataset collected from four areas in the real environment. Extensive experiments conducted on our datasets demonstrate the effectiveness and quality of the proposed synthetic dataset.

Figures (8)

• Figure 1: Example point clouds in STPLS3D dataset. Top row: synthetic point clouds with point-wise semantic and instance annotations. Bottom row: real point clouds captured from USC.
• Figure 2: The proposed synthetic data generation pipeline.
• Figure 3: Examples of our STPLS3D dataset, including the proposed Synthetic V1, Synthetic V2, Synthetic V3, and the real-world subsets. Different semantic classes are shown in different colors, as illustrated in the color legend. Note that different instances are displayed in different random colors. Best viewed in color.
• Figure 4: The class distribution of real-dataset of our STPLS3D. Note the logarithmic scale for the vertical axis.
• Figure 5: The class distribution of synthetic subsets of our STPLS3D. Note the logarithmic scale for the vertical axis.