Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Illicit object detection in X-ray images using Vision Transformers

Jorgen Cani, Ioannis Mademlis, Adamantia Anna Rebolledo Chrysochoou, Georgios Th. Papadopoulos

TL;DR

The paper tackles illicit object detection in security-relevant X-ray images by systematically evaluating Vision Transformer backbones (SWIN, NextViT) and detection heads (DINO, RT-DETR) alongside a CNN baseline (YOLOv8). Using pretraining on SIXray and fine-tuning on CFRay, the study compares accuracy and inference speed, highlighting that NextViT with YOLOv8 achieves real-time performance, while SWIN paired with DINO attains strong accuracy at higher IoU thresholds. DINO demonstrates strong performance in low-data regimes, and hybrid CNN–Transformer configurations outperform pure end-to-end Transformers in domain-specific scenarios. The findings provide practical guidance on selecting Transformer-based versus hybrid architectures for X-ray screening throughput and reliability, and suggest avenues for integrating domain-specific modules to further boost accuracy. Overall, the work informs deployment decisions in high-security environments where both speed and precise localization of illicit items are critical.

Abstract

Illicit object detection is a critical task performed at various high-security locations, including airports, train stations, subways, and ports. The continuous and tedious work of examining thousands of X-ray images per hour can be mentally taxing. Thus, Deep Neural Networks (DNNs) can be used to automate the X-ray image analysis process, improve efficiency and alleviate the security officers' inspection burden. The neural architectures typically utilized in relevant literature are Convolutional Neural Networks (CNNs), with Vision Transformers (ViTs) rarely employed. In order to address this gap, this paper conducts a comprehensive evaluation of relevant ViT architectures on illicit item detection in X-ray images. This study utilizes both Transformer and hybrid backbones, such as SWIN and NextViT, and detectors, such as DINO and RT-DETR. The results demonstrate the remarkable accuracy of the DINO Transformer detector in the low-data regime, the impressive real-time performance of YOLOv8, and the effectiveness of the hybrid NextViT backbone.

Illicit object detection in X-ray images using Vision Transformers

TL;DR

The paper tackles illicit object detection in security-relevant X-ray images by systematically evaluating Vision Transformer backbones (SWIN, NextViT) and detection heads (DINO, RT-DETR) alongside a CNN baseline (YOLOv8). Using pretraining on SIXray and fine-tuning on CFRay, the study compares accuracy and inference speed, highlighting that NextViT with YOLOv8 achieves real-time performance, while SWIN paired with DINO attains strong accuracy at higher IoU thresholds. DINO demonstrates strong performance in low-data regimes, and hybrid CNN–Transformer configurations outperform pure end-to-end Transformers in domain-specific scenarios. The findings provide practical guidance on selecting Transformer-based versus hybrid architectures for X-ray screening throughput and reliability, and suggest avenues for integrating domain-specific modules to further boost accuracy. Overall, the work informs deployment decisions in high-security environments where both speed and precise localization of illicit items are critical.

Abstract

Illicit object detection is a critical task performed at various high-security locations, including airports, train stations, subways, and ports. The continuous and tedious work of examining thousands of X-ray images per hour can be mentally taxing. Thus, Deep Neural Networks (DNNs) can be used to automate the X-ray image analysis process, improve efficiency and alleviate the security officers' inspection burden. The neural architectures typically utilized in relevant literature are Convolutional Neural Networks (CNNs), with Vision Transformers (ViTs) rarely employed. In order to address this gap, this paper conducts a comprehensive evaluation of relevant ViT architectures on illicit item detection in X-ray images. This study utilizes both Transformer and hybrid backbones, such as SWIN and NextViT, and detectors, such as DINO and RT-DETR. The results demonstrate the remarkable accuracy of the DINO Transformer detector in the low-data regime, the impressive real-time performance of YOLOv8, and the effectiveness of the hybrid NextViT backbone.
Paper Structure (15 sections, 5 equations, 3 figures, 3 tables)

This paper contains 15 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Object Detection Methods
  4. Backbone Networks
  5. Detection Head Networks
  6. Experimental Evaluation
  7. Experimental Datasets
  8. Evaluation Metrics
  9. Detection Accuracy Metrics
  10. Speed Metric
  11. Experimental Setup
  12. Evaluation Results
  13. Ablation Studies
  14. Conclusions
  15. Acknowledgment

Figures (3)

  • Figure 1: The architecture of the NextViT-s backbone. Image from liNextViTNextGeneration2022.
  • Figure 2: The DINO architecture. Image from zhangDINODETRImproved2022.
  • Figure 3: A sample from the CFray dataset.