Multi Modal Attention Networks with Uncertainty Quantification for Automated Concrete Bridge Deck Delamination Detection

TL;DR

This work tackles automated bridge deck delamination detection by fusing GPR and IRT data with a multi‑modal attention network that includes temporal GPR attention, spatial IRT attention, and cross‑modal fusion via learnable modality embeddings. Uncertainty is quantified through Monte Carlo dropout and a learned variance head, decomposing toward epistemic and aleatoric components to support risk‑aware decisions. Across five SDNET2021 bridges, the approach yields substantial accuracy and AUC gains over single‑modal and simple fusion baselines, with strong calibration improvements (ECE reduced by a large margin) and robust performance on balanced to moderately imbalanced datasets; however, extreme class imbalance reveals limitations and guides practitioners toward specialized techniques. The findings provide actionable deployment guidance for real‑world inspection, including real‑time capability (14.8M parameters, ~28 ms inference) and reliable uncertainty estimates to selectively route uncertain cases for human review.

Abstract

Deteriorating civil infrastructure requires automated inspection techniques overcoming limitations of visual assessment. While Ground Penetrating Radar and Infrared Thermography enable subsurface defect detection, single modal approaches face complementary constraints radar struggles with moisture and shallow defects, while thermography exhibits weather dependency and limited depth. This paper presents a multi modal attention network fusing radar temporal patterns with thermal spatial signatures for bridge deck delamination detection. Our architecture introduces temporal attention for radar processing, spatial attention for thermal features, and cross modal fusion with learnable embeddings discovering complementary defect patterns invisible to individual sensors. We incorporate uncertainty quantification through Monte Carlo dropout and learned variance estimation, decomposing uncertainty into epistemic and aleatoric components for safety critical decisions. Experiments on five bridge datasets reveal that on balanced to moderately imbalanced data, our approach substantially outperforms baselines in accuracy and AUC representing meaningful improvements over single modal and concatenation based fusion. Ablation studies demonstrate cross modal attention provides critical gains beyond within modality attention, while multi head mechanisms achieve improved calibration. Uncertainty quantification reduces calibration error, enabling selective prediction by rejecting uncertain cases. However, under extreme class imbalance, attention mechanisms show vulnerability to majority class collapse. These findings provide actionable guidance: attention based architecture performs well across typical scenarios, while extreme imbalance requires specialized techniques. Our system maintains deployment efficiency, enabling real time inspection with characterized capabilities and limitations.