SegTrans: Transferable Adversarial Examples for Segmentation Models

TL;DR

SegTrans tackles the transferability gap of adversarial attacks on semantic segmentation by leveraging partial semantic information through a two-module design: multi-region perturbation activation and semantic remapping. By preserving and reassembling regional semantic cues, it reduces dependence on surrogate-model features and disrupts contextual coherence, achieving superior cross-model transferability. Extensive experiments across four segmentation models and two datasets show a notable average gain of $+8.55\%$ in transfer attack success rate with competitive efficiency, and robustness against several defenses. The approach highlights a practical vulnerability of segmentation systems to region-based perturbations and offers a framework for both advancing attack research and informing defense strategies.

Abstract

Segmentation models exhibit significant vulnerability to adversarial examples in white-box settings, but existing adversarial attack methods often show poor transferability across different segmentation models. While some researchers have explored transfer-based adversarial attack (i.e., transfer attack) methods for segmentation models, the complex contextual dependencies within these models and the feature distribution gaps between surrogate and target models result in unsatisfactory transfer success rates. To address these issues, we propose SegTrans, a novel transfer attack framework that divides the input sample into multiple local regions and remaps their semantic information to generate diverse enhanced samples. These enhanced samples replace the original ones for perturbation optimization, thereby improving the transferability of adversarial examples across different segmentation models. Unlike existing methods, SegTrans only retains local semantic information from the original input, rather than using global semantic information to optimize perturbations. Extensive experiments on two benchmark datasets, PASCAL VOC and Cityscapes, four different segmentation models, and three backbone networks show that SegTrans significantly improves adversarial transfer success rates without introducing additional computational overhead. Compared to the current state-of-the-art methods, SegTrans achieves an average increase of 8.55% in transfer attack success rate and improves computational efficiency by more than 100%.

Figures (10)

• Figure 1: Illustration of transfer-based adversarial attacks
• Figure 2: The perturbation failure caused by tight coupling phenomenon between objects. Vanilla and Vanilla w/o BG represent the attack results of vanilla on the original image and the attack results of vanilla after removing the background outside the target, respectively. Here, "Vanilla" refers to the baseline attack method PGD madry2018towards that we have selected.
• Figure 3: Grad-CAM visualizations of different models on the same input image, where each row represents the same sample. FCN serves as the surrogate model, while the other three are target models. Global semantic features, DeepLabV1 and DeepLabV3+ are abbreviated as G-S features, DV1 and DV3+, respectively.
• Figure 4: The framework of SegTrans
• Figure 5: The visualization results of adversarial examples by SegTrans attack