Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3D Semantic Segmentation for Post-Disaster Assessment

Nhut Le, Maryam Rahnemoonfar

TL;DR

This paper addresses the challenge of 3D semantic segmentation for post-disaster assessment by constructing a disaster-specific 3D dataset from UAV footage of Hurricane Ian using Structure-from-Motion and Multi-View Stereo, with ground-truth labels generated from 2D annotations. It evaluates state-of-the-art 3D segmentation models—Fast Point Transformer, Point Transformer v3, and OA-CNNs—on the new dataset and finds significant limitations when handling disaster-specific outdoor scenes, particularly in distinguishing damaged versus undamaged buildings. The work highlights a critical gap in available benchmarks for post-disaster 3D understanding and demonstrates the need for specialized datasets and model innovations tailored to disaster environments. Overall, the study provides a valuable dataset, a rigorous benchmark for post-disaster 3D semantic segmentation, and a clear roadmap for developing robust methods with practical rescue and response implications.

Abstract

The increasing frequency of natural disasters poses severe threats to human lives and leads to substantial economic losses. While 3D semantic segmentation is crucial for post-disaster assessment, existing deep learning models lack datasets specifically designed for post-disaster environments. To address this gap, we constructed a specialized 3D dataset using unmanned aerial vehicles (UAVs)-captured aerial footage of Hurricane Ian (2022) over affected areas, employing Structure-from-Motion (SfM) and Multi-View Stereo (MVS) techniques to reconstruct 3D point clouds. We evaluated the state-of-the-art (SOTA) 3D semantic segmentation models, Fast Point Transformer (FPT), Point Transformer v3 (PTv3), and OA-CNNs on this dataset, exposing significant limitations in existing methods for disaster-stricken regions. These findings underscore the urgent need for advancements in 3D segmentation techniques and the development of specialized 3D benchmark datasets to improve post-disaster scene understanding and response.

3D Semantic Segmentation for Post-Disaster Assessment

TL;DR

This paper addresses the challenge of 3D semantic segmentation for post-disaster assessment by constructing a disaster-specific 3D dataset from UAV footage of Hurricane Ian using Structure-from-Motion and Multi-View Stereo, with ground-truth labels generated from 2D annotations. It evaluates state-of-the-art 3D segmentation models—Fast Point Transformer, Point Transformer v3, and OA-CNNs—on the new dataset and finds significant limitations when handling disaster-specific outdoor scenes, particularly in distinguishing damaged versus undamaged buildings. The work highlights a critical gap in available benchmarks for post-disaster 3D understanding and demonstrates the need for specialized datasets and model innovations tailored to disaster environments. Overall, the study provides a valuable dataset, a rigorous benchmark for post-disaster 3D semantic segmentation, and a clear roadmap for developing robust methods with practical rescue and response implications.

Abstract

The increasing frequency of natural disasters poses severe threats to human lives and leads to substantial economic losses. While 3D semantic segmentation is crucial for post-disaster assessment, existing deep learning models lack datasets specifically designed for post-disaster environments. To address this gap, we constructed a specialized 3D dataset using unmanned aerial vehicles (UAVs)-captured aerial footage of Hurricane Ian (2022) over affected areas, employing Structure-from-Motion (SfM) and Multi-View Stereo (MVS) techniques to reconstruct 3D point clouds. We evaluated the state-of-the-art (SOTA) 3D semantic segmentation models, Fast Point Transformer (FPT), Point Transformer v3 (PTv3), and OA-CNNs on this dataset, exposing significant limitations in existing methods for disaster-stricken regions. These findings underscore the urgent need for advancements in 3D segmentation techniques and the development of specialized 3D benchmark datasets to improve post-disaster scene understanding and response.
Paper Structure (18 sections, 3 figures, 2 tables)

This paper contains 18 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets and Benchmarks for Damage Assessment
  4. 3D Benchmark Datasets
  5. 3D Indoor Benchmark Datasets
  6. 3D Outdoor Benchmark Datasets
  7. Methodology
  8. 3D Semantic Segmentation Methods
  9. Our 3D Dataset
  10. 3D Reconstruction
  11. Ground Truth Generation
  12. Dataset Splits
  13. Experiment and Result
  14. Metric
  15. Evaluation our 3D dataset
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Point clouds in our dataset with semantic labels: Cyan – Building (no damage), Red – Building (damaged), Yellow – Road, Green – Tree, Black – Background
  • Figure 2: The reconstruction pipeline. Frames extracted from UAV aerial footage are processed using Structure-from-Motion (SfM) and Multi-View Stereo (MVS) to generate 3D point clouds. Outliers are manually removed to obtain clean and accurate reconstructions.
  • Figure 3: 3D Sem. Seg. Generation. 2D annotations were projected into 3D space using majority voting across multiple frames, assigning each point the most frequent label from its 2D projections. Final refinements were made in CloudCompare cloudcompare to ensure accurate point-level segmentation.