TrackPGD: Efficient Adversarial Attack using Object Binary Masks against Robust Transformer Trackers

TL;DR

This work introduces TrackPGD, a novel white-box attack that utilizes predicted object binary masks to target robust transformer trackers and addresses two primary challenges in adapting a segmentation attack for trackers: limited class numbers and extreme pixel class imbalance.

Abstract

Adversarial perturbations can deceive neural networks by adding small, imperceptible noise to the input. Recent object trackers with transformer backbones have shown strong performance on tracking datasets, but their adversarial robustness has not been thoroughly evaluated. While transformer trackers are resilient to black-box attacks, existing white-box adversarial attacks are not universally applicable against these new transformer trackers due to differences in backbone architecture. In this work, we introduce TrackPGD, a novel white-box attack that utilizes predicted object binary masks to target robust transformer trackers. Built upon the powerful segmentation attack SegPGD, our proposed TrackPGD effectively influences the decisions of transformer-based trackers. Our method addresses two primary challenges in adapting a segmentation attack for trackers: limited class numbers and extreme pixel class imbalance. TrackPGD uses the same number of iterations as other attack methods for tracker networks and produces competitive adversarial examples that mislead transformer and non-transformer trackers such as MixFormerM, OSTrackSTS, TransT-SEG, and RTS on datasets including VOT2022STS, DAVIS2016, UAV123, and GOT-10k.

Figures (5)

• Figure 1: The binary masks generated by MixFormerM cui_MixFormer_2022 and OSTrackSTS ye_joint_2022 attacked by our TrackPGD vs. IoU jia_iou_2021 which is the most recent and applicable attack for these two trackers up to now. The green/red masks are the before/after the attack outputs.
• Figure 2: The before (green) and after attack masks (red) generated by MixFormerM cui_MixFormer_2022, while the vanilla SegPGDs and difference loss are used instead of binary cross entropy in the focal loss for the TrackPGD attack.
• Figure 3: The TrackPGD parameter heatmaps from left to right, first row: MixFormerMcui_MixFormer_2022yan_alpha-refine_2021, OSTrackSTS ye_joint_2022yan_alpha-refine_2021 and second row: TransT-SEG chen_high-performance_2023, RTS paul_robust_2022 trackers.
• Figure 4: The results of MixFormerM cui_MixFormer_2022, RTS paul_robust_2022, and OSTrackSTS ye_joint_2022 attacked by TrackPGD. The green mask is the original output, while the red mask is the after attack mask.
• Figure 5: The TransT-SEG chen_high-performance_2023 tracker after SPARK guo_spark_2020, RTAA jia_robust_2020, CSA yan_cooling-shrinking_2020, IoU jia_iou_2021 and TrackPGD. The green/red colors correspond to before and after the attack.