Architectural Insights for Post-Tornado Damage Recognition

TL;DR

This work tackles the challenge of rapid post-tornado damage recognition from ground-level imagery, a domain plagued by severe visual shift and data imbalance. It introduces the Quad-State Tornado Damage (QSTD) benchmark and systematically evaluates 79 open-source architectures (CNNs and Vision Transformers) across more than 2,300 experiments to disentangle architectural effects from optimization strategies. A central finding is that optimizer choice can be more influential than architecture for Transformer families, with a universal benefit from a learning rate of $1\times 10^{-4}$ and notable gains when using SGD over Adam. The champion approach (ConvNeXt-Base trained with SGD at $1\times 10^{-4}$) generalizes well to the Tuscaloosa-Moore TMTD dataset in a zero-shot setting, achieving $46.4\\%$ Macro F1 and $85.5\\%$ Ordinal Top-1 Accuracy, underscoring the value of co-optimized architectures and training dynamics for robust cross-event deployment in disaster response.

Abstract

Rapid and accurate building damage assessment in the immediate aftermath of tornadoes is critical for coordinating life-saving search and rescue operations, optimizing emergency resource allocation, and accelerating community recovery. However, current automated methods struggle with the unique visual complexity of tornado-induced wreckage, primarily due to severe domain shift from standard pre-training datasets and extreme class imbalance in real-world disaster data. To address these challenges, we introduce a systematic experimental framework evaluating 79 open-source deep learning models, encompassing both Convolutional Neural Networks (CNNs) and Vision Transformers, across over 2,300 controlled experiments on our newly curated Quad-State Tornado Damage (QSTD) benchmark dataset. Our findings reveal that achieving operational-grade performance hinges on a complex interaction between architecture and optimization, rather than architectural selection alone. Most strikingly, we demonstrate that optimizer choice can be more consequential than architecture: switching from Adam to SGD provided dramatic F1 gains of +25 to +38 points for Vision Transformer and Swin Transformer families, fundamentally reversing their ranking from bottom-tier to competitive with top-performing CNNs. Furthermore, a low learning rate of 1x10^(-4) proved universally critical, boosting average F1 performance by +10.2 points across all architectures. Our champion model, ConvNeXt-Base trained with these optimized settings, demonstrated strong cross-event generalization on the held-out Tuscaloosa-Moore Tornado Damage (TMTD) dataset, achieving 46.4% Macro F1 (+34.6 points over baseline) and retaining 85.5% Ordinal Top-1 Accuracy despite temporal and sensor domain shifts.

Figures (19)

• Figure 1: Example of QSTD images for the 6-class IN-CORE taxonomy
• Figure 2: (a) Class distribution of the QSTD dataset across training, validation, and test splits, highlighting the pronounced long-tail imbalance. (b) Overall class distribution of the TMTD dataset used for cross-event evaluation
• Figure 3: Example of TMTD images for the 6-class IN-CORE taxonomy
• Figure 4: The four-stage experimental design, from zero-shot baseline experiment to cross-event generalisation evaluation.
• Figure 5: (a) Zero-Shot Transfer Performance by Pre-training Source. (b) Scatter Plot of Macro F1 vs. Parameters for all 21 zero-shot models showing no correlation between parameter count and F1.