CBAM-SwinT-BL: Small Rail Surface Defect Detection Method Based on Swin Transformer with Block Level CBAM Enhancement

Abstract

Under high-intensity rail operations, rail tracks endure considerable stresses resulting in various defects such as corrugation and spellings. Failure to effectively detect defects and provide maintenance in time would compromise service reliability and public safety. While advanced models have been developed in recent years, efficiently identifying small-scale rail defects has not yet been studied, especially for categories such as Dirt or Squat on rail surface. To address this challenge, this study utilizes Swin Transformer (SwinT) as baseline and incorporates the Convolutional Block Attention Module (CBAM) for enhancement. Our proposed method integrates CBAM successively within the swin transformer blocks, resulting in significant performance improvement in rail defect detection, particularly for categories with small instance sizes. The proposed framework is named CBAM-Enhanced Swin Transformer in Block Level (CBAM-SwinT-BL). Experiment and ablation study have proven the effectiveness of the framework. The proposed framework has a notable improvement in the accuracy of small size defects, such as dirt and dent categories in RIII dataset, with mAP-50 increasing by +23.0% and +38.3% respectively, and the squat category in MUET dataset also reaches +13.2% higher than the original model. Compares to the original SwinT, CBAM-SwinT-BL increase overall precision around +5% in the MUET dataset and +7% in the RIII dataset, reaching 69.1% and 88.1% respectively. Meanwhile, the additional module CBAM merely extend the model training speed by an average of +0.04s/iteration, which is acceptable compared to the significant improvement in system performance.

Figures (17)

• Figure 1: CBAM module architecture. (a) Channel attention module. (b) Spatial attention module.
• Figure 2: The architecture of Swin Transformer model. (a) General structure of Swin Transformer. (b) two successive Swin Transformer blocks.
• Figure 3: CBAM enhanced Swin Transformer framework. (a) CBAM in Model Level. (b) CBAM in Stage Level. (c) Flowchart of Patch Merging in Stage Level. (d) CBAM in Block Level. (e) Flowchart of CAM and SAM sequentially located inside Swin Transformer Blocks.
• Figure 4: Samples of the MUET dataset with 7 categories. From left to right, the first row contains Crack, Flaking, Groove and Joint, and the second row contains Shelling, Spalling and Squat.
• Figure 5: Samples of the RIII dataset with 8 categories. From left to right, the first row contains Damage, Dirt, Unknown and Gap, and the second row contains Dent, Crush, Scratch and Slant.