Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling

Abstract

In complex environments, infrared object detection exhibits broad applicability and stability across diverse scenarios. However, infrared object detection is vulnerable to both common corruptions and adversarial examples, leading to potential security risks. To improve the robustness of infrared object detection, current methods mostly adopt a data-driven ideology, which only superficially drives the network to fit the training data without specifically considering the unique characteristics of infrared images, resulting in limited robustness. In this paper, we revisit infrared physical knowledge and find that relative thermal radiation relations between different classes can be regarded as a reliable knowledge source under the complex scenarios of adversarial examples and common corruptions. Thus, we theoretically model thermal radiation relations based on the rank order of gray values for different classes, and further quantify the stability of various inter-class thermal radiation relations. Based on the above theoretical framework, we propose Knowledge-Guided Adversarial Training (KGAT) for infrared object detection, in which infrared physical knowledge is embedded into the adversarial training process, and the predicted results are optimized to be consistent with the actual physical laws. Extensive experiments on three infrared datasets and six mainstream infrared object detection models demonstrate that KGAT effectively enhances both clean accuracy and robustness against adversarial attacks and common corruptions.

Figures (6)

• Figure 1: Performance of the baseline and our knowledge-guided adversarial training (KGAT) on M$^{3}$FD dataset, using YOLO-v8 as the detector. Compared with Data-driven method, KGAT can enhance the robustness of infrared object detection towards more diverse perturbations in complex infrared environments.
• Figure 2: The advantage of relative thermal radiation relations. In the M$^3$FD dataset, under environmental conditions such as Clear Day, Cloudy Overcast, and Clear Dusk, although the absolute gray values of the "car" and the "people" are different, most of the relative thermal radiation relations between "car" and the "people" keep stable.
• Figure 3: The Framework of our Knowledge-Guided Adversarial Training.During the knowledge extraction process, we first calculate the rank order of the gray value between different classes to extract the thermal radiation relation. We further calculate the stability of the thermal radiation relation between all the different classes. During the training process, we apply the thermal radiation relations and the corresponding variations to adjust the learning weights of each training sample so that the prediction results conform to the real-world infrared knowledge.
• Figure 4: Visualization of the prediction results between baseline (SALC) and our SALC+KGAT of YOLO-v8 on M$^3$FD under three different attacks.
• Figure 5: The robustness of YOLO-v8 based SALC and SALC+KGAT (ours) against A$_{mtd}$ attack on images with thermal radiation relations of different stability. In the x-axis labels, C, P, B, M, L, and T denote Cars, People, Bus, Motorcycle, Lamp, and Truck, respectively. The x-axis denotes the specific thermal radiation relation, and the bars indicate the stability level of that relation, using the left y-axis. The two curves represent the mAP$_{50}$ of SALC and SALC+KGAT (ours) on images containing this specific thermal radiation relation, respectively, using the right y-axis.

Theorems & Definitions (2)

• Corollary 1
• Theorem 1