Imbalance-Aware Culvert-Sewer Defect Segmentation Using an Enhanced Feature Pyramid Network

TL;DR

This work tackles the challenge of imbalanced defect data in culvert and sewer segmentation by proposing the Enhanced Feature Pyramid Network (E-FPN), a dual-pathway architecture that combines a sparsely connected, Inception-like bottom-up block with depth-wise separable convolutions and a high-resolution top-down fusion path. It further mitigates imbalance through class decomposition and targeted data augmentation, validated on the Culvert-Sewer Defects dataset and an aerial drone dataset, achieving substantial IoU gains and dramatically reduced parameter counts. Key contributions include a rigorous ablation study showing the benefit of multi-scale depthwise blocks, and the demonstration that combining CD and DA yields the largest performance gains ($ ext{IoU}$ improvement of $≈6.97 ext%$ on average). The model offers a practical, scalable solution for infrastructure inspection, with implications for broader multi-class segmentation in imbalanced real-world datasets and potential extensions to temporal, unsupervised, and physics-informed approaches.

Abstract

Imbalanced datasets are a significant challenge in real-world scenarios. They lead to models that underperform on underrepresented classes, which is a critical issue in infrastructure inspection. This paper introduces the Enhanced Feature Pyramid Network (E-FPN), a deep learning model for the semantic segmentation of culverts and sewer pipes within imbalanced datasets. The E-FPN incorporates architectural innovations like sparsely connected blocks and depth-wise separable convolutions to improve feature extraction and handle object variations. To address dataset imbalance, the model employs strategies like class decomposition and data augmentation. Experimental results on the culvert-sewer defects dataset and a benchmark aerial semantic segmentation drone dataset show that the E-FPN outperforms state-of-the-art methods, achieving an average Intersection over Union (IoU) improvement of 13.8% and 27.2%, respectively. Additionally, class decomposition and data augmentation together boost the model's performance by approximately 6.9% IoU. The proposed E-FPN presents a promising solution for enhancing object segmentation in challenging, multi-class real-world datasets, with potential applications extending beyond culvert-sewer defect detection.

Figures (7)

• Figure 1: E-FPN architecture: Dual-pathway design for multi-scale feature extraction. The bottom-up pathway filters and down-samples the input image by a factor of 2 at each layer using enhanced Inception blocks with depth-wise separable convolutions. The top-down pathway employs upsampling and feature fusion to reconstruct a colored-masked image. Numbers indicate feature map dimensions and channel depths at each stage.
• Figure 2: Deficiency distribution in the Culvert-Sewer Defects Dataset. The dataset exhibits significant imbalance, with sample counts ranging from 2,340 in the highest class to 104 in the lowest class.
• Figure 3: Workflow for Mitigating Class Imbalance through Class Decomposition and Data Augmentation Techniques. The figure illustrates the process of applying class decomposition to group similar classes and the targeted data augmentation to balance the dataset. Models trained on these balanced samples have improved performance and generalization.
• Figure 4: Comparative segmentation results on the culvert-sewer defects dataset are shown, with the first row illustrating pipe deformation defects, the second row showing a crack, and the third row illustrating joint misalignment: (a) Original images (b) Ground truth (c) U-Net (d) CBAM U-Net (e) FPN with ResNet (f) ASCU-Net (g) Swin Transformer, (h) E-FPN (this paper)
• Figure 5: Comparative validation metrics of E-FPN against baseline and state-of-the-art models on the culvert-sewer defects dataset: (a) Cross-entropy loss, (b) F1-score, and (c) IoU. The proposed model in the blue color shows the highest validation IoU and F1-score compared to the other models.