Risk Assessment for Autonomous Landing in Urban Environments using Semantic Segmentation

TL;DR

The paper tackles risk-aware autonomous landing in urban environments by combining SegFormer-based semantic segmentation with a six-level risk clustering aligned to SORA to identify Safe Landing Zones. It finetunes SegFormer on the Semantic Drone Dataset to produce pixel-level urban scene labels, then maps these labels to risk levels to guide emergency landing decisions. The approach runs in real time on onboard hardware (approximately 14 FPS on a Jetson Xavier) and is validated through case studies demonstrating robustness and potential to improve regulatory compliance. While high-level risk levels achieve strong accuracy, some confusion remains for mid-to-high risk classes, indicating avenues for dataset expansion and altitude-aware confidence indexing. Overall, the framework provides a practical, regulation-aligned method to enhance safety for urban UAV operations and emergency landing decision-making.

Abstract

In this paper, we address the vision-based autonomous landing problem in complex urban environments using deep neural networks for semantic segmentation and risk assessment. We propose employing the SegFormer, a state-of-the-art visual transformer network, for the semantic segmentation of complex, unstructured urban environments. This approach yields valuable information that can be utilized in smart autonomous landing missions, particularly in emergency landing scenarios resulting from system failures or human errors. The assessment is done in real-time flight, when images of an RGB camera at the Unmanned Aerial Vehicle (UAV) are segmented with the SegFormer into the most common classes found in urban environments. These classes are then mapped into a level of risk, considering in general, potential material damage, damaging the drone itself and endanger people. The proposed strategy is validated through several case studies, demonstrating the huge potential of semantic segmentation-based strategies to determining the safest landing areas for autonomous emergency landing, which we believe will help unleash the full potential of UAVs on civil applications within urban areas.

Figures (6)

• Figure 1: SegFormer architecture proposed by Xie et al.xie2021segformer.
• Figure 2: Samples of aerial images in the Semantic Drone Dataset, captured in urban environments, using a down-looking camera mounted on a UAV. These images are labeled with several common classes found in urban areas, including grass, pavement, people, cars, vegetation, etc.
• Figure 3: The process of inference and risk assessment: first image shows the input image, the second is the ground truth reference of semantic segmentation, the third picture shows the prediction inferred from model (colored for visual interpretation), and last image presents the mapping to risk levels, where red regions are the riskiest and blue the safest. Reference for colors in Table \ref{['tab:classes']}.
• Figure 4: Confusion matrix of the testing set on the six risk levels.
• Figure 5: Experimental results obtained for $5$ different case studies within the test set. The first column presents the original image, the ground truth annotated image is depicted in the second column, and the third column contains the inferred semantic segmentation. The last column shows the risk levels, where the red color represents the highest risk, passing through orange, yellow, green, cyan, to blue, which represents the safest areas. Reference for colors in Table \ref{['tab:classes']}.