TUM2TWIN: Introducing the Large-Scale Multimodal Urban Digital Twin Benchmark Dataset

TL;DR

TUM2TWIN addresses the lack of comprehensive, multimodal, georeferenced benchmarks for Urban Digital Twins by introducing a large-scale dataset that integrates point clouds, imagery, networks, and 3D models across 32 subsets over ~100,000 m^2. The approach provides end-to-end data fidelity and cross-representation ground-truth, enabling robust validation of reconstruction, registration, and synthesis methods, including NeRF and Gaussian Splatting, as well as solar potential analysis. Key contributions include high-accuracy georeferenced TLS/MLS/UAS/ALS data, diverse image modalities, HD maps, CityGML-aligned semantic models at LoD1–LoD3, and multi-modal coregistration experiments. The dataset is poised to accelerate practical UDT development and cross-domain research by offering a unified, extensible platform for urban sensing, modeling, and reasoning across academia and industry.

Abstract

Urban Digital Twins (UDTs) have become essential for managing cities and integrating complex, heterogeneous data from diverse sources. Creating UDTs involves challenges at multiple process stages, including acquiring accurate 3D source data, reconstructing high-fidelity 3D models, maintaining models' updates, and ensuring seamless interoperability to downstream tasks. Current datasets are usually limited to one part of the processing chain, hampering comprehensive UDTs validation. To address these challenges, we introduce the first comprehensive multimodal Urban Digital Twin benchmark dataset: TUM2TWIN. This dataset includes georeferenced, semantically aligned 3D models and networks along with various terrestrial, mobile, aerial, and satellite observations boasting 32 data subsets over roughly 100,000 $m^2$ and currently 767 GB of data. By ensuring georeferenced indoor-outdoor acquisition, high accuracy, and multimodal data integration, the benchmark supports robust analysis of sensors and the development of advanced reconstruction methods. Additionally, we explore downstream tasks demonstrating the potential of TUM2TWIN, including novel view synthesis of NeRF and Gaussian Splatting, solar potential analysis, point cloud semantic segmentation, and LoD3 building reconstruction. We are convinced this contribution lays a foundation for overcoming current limitations in UDT creation, fostering new research directions and practical solutions for smarter, data-driven urban environments. The project is available under: https://tum2t.win

Figures (21)

• Figure 1: TUM2TWIN: Georeferenced, semantic-rich, multimodal, multitemporal, and high-fidelity benchmark dataset for the development of urban digital twins (UDT).
• Figure 2: TUM2TWIN covers approximately 100 000 $m^2$ of the center of Munich, Germany (highlighted), boasting 32 data subsets and currently totaling 767 GB.
• Figure 3: Timeline of the TUM2TWIN benchmark dataset.
• Figure 4: Data dependency graph presenting the relation of the TUM2TWIN sub-datasets. Each data is bound to the spatial and temporal accuracy of the source data. The release data can differ to the acquisition date as marked by asterisk ($*$).
• Figure 5: TUM2TWIN data acquisition campaigns (w/o derived high-level representations of Networks and 3D Models).