X-ray illicit object detection using hybrid CNN-transformer neural network architectures

TL;DR

The paper investigates hybrid CNN-transformer architectures for illicit object detection in X-ray security imaging, addressing occlusion and cross-scanner domain shifts. By forming detectors with backbones HGNetV2 or Next-ViT-S and heads YOLOv8 or RT-DETR, the study benchmarks performance on EDS, HiXray, and PIDray datasets. Results indicate that CNN-only detectors excel on HiXray and PIDray, while hybrid models offer robustness to distribution shifts in EDS, with Next-ViT-S combined with YOLOv8 often providing superior results. The work highlights the importance of architecture-head compatibility and provides guidelines for future exploration, including broader datasets and additional hybrid configurations.

Abstract

In the field of X-ray security applications, even the smallest details can significantly impact outcomes. Objects that are heavily occluded or intentionally concealed pose a great challenge for detection, whether by human observation or through advanced technological applications. While certain Deep Learning (DL) architectures demonstrate strong performance in processing local information, such as Convolutional Neural Networks (CNNs), others excel in handling distant information, e.g., transformers. In X-ray security imaging the literature has been dominated by the use of CNN-based methods, while the integration of the two aforementioned leading architectures has not been sufficiently explored. In this paper, various hybrid CNN-transformer architectures are evaluated against a common CNN object detection baseline, namely YOLOv8. In particular, a CNN (HGNetV2) and a hybrid CNN-transformer (Next-ViT-S) backbone are combined with different CNN/transformer detection heads (YOLOv8 and RT-DETR). The resulting architectures are comparatively evaluated on three challenging public X-ray inspection datasets, namely EDS, HiXray, and PIDray. Interestingly, while the YOLOv8 detector with its default backbone (CSP-DarkNet53) is generally shown to be advantageous on the HiXray and PIDray datasets, when a domain distribution shift is incorporated in the X-ray images (as happens in the EDS datasets), hybrid CNN-transformer architectures exhibit increased robustness. Detailed comparative evaluation results, including object-level detection performance and object-size error analysis, demonstrate the strengths and weaknesses of each architectural combination and suggest guidelines for future research. The source code and network weights of the models employed in this study are available at https://github.com/jgenc/xray-comparative-evaluation.

Figures (5)

• Figure 1: Architecture of the D(YOLOv8, Next-ViT-S) detector.
• Figure 2: Architecture of the D(RT-DETR, Next-ViT-S) detector.
• Figure 3: Object-level performance (mAP$^{50:95}$ metric) for datasets: a) EDS, b) HiXray, and c) PIDray.
• Figure 4: Object scale-related performance (mAP$^{50:95}$ metric) for datasets: a) EDS, b) HiXray, and c) PIDray.
• Figure 5: Indicative object detection results in the EDS, HiXray, and PIDray datasets.