Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Erosion Attack for Adversarial Training to Enhance Semantic Segmentation Robustness

Yufei Song, Ziqi Zhou, Menghao Deng, Yifan Hu, Shengshan Hu, Minghui Li, Leo Yu Zhang

TL;DR

Semantic segmentation models are vulnerable to adversarial perturbations, risking critical applications. The authors propose EroSeg-AT, a vulnerability-aware adversarial training framework that first targets low-confidence pixels and then progressively propagates perturbations to disrupt contextual semantic consistency. EroSeg comprises confidence-based pixel selection and progressive perturbation propagation, with a formal min-max AT objective and pixel-level weighting to emphasize foreground regions. Experiments on DeepLabV3 and PSPNet across Pascal VOC and Cityscapes show stronger attack performance than baselines and improved robustness under AT, offering a new avenue for defenses in dense prediction tasks.

Abstract

Existing segmentation models exhibit significant vulnerability to adversarial attacks.To improve robustness, adversarial training incorporates adversarial examples into model training. However, existing attack methods consider only global semantic information and ignore contextual semantic relationships within the samples, limiting the effectiveness of adversarial training. To address this issue, we propose EroSeg-AT, a vulnerability-aware adversarial training framework that leverages EroSeg to generate adversarial examples. EroSeg first selects sensitive pixels based on pixel-level confidence and then progressively propagates perturbations to higher-confidence pixels, effectively disrupting the semantic consistency of the samples. Experimental results show that, compared to existing methods, our approach significantly improves attack effectiveness and enhances model robustness under adversarial training.

Erosion Attack for Adversarial Training to Enhance Semantic Segmentation Robustness

TL;DR

Semantic segmentation models are vulnerable to adversarial perturbations, risking critical applications. The authors propose EroSeg-AT, a vulnerability-aware adversarial training framework that first targets low-confidence pixels and then progressively propagates perturbations to disrupt contextual semantic consistency. EroSeg comprises confidence-based pixel selection and progressive perturbation propagation, with a formal min-max AT objective and pixel-level weighting to emphasize foreground regions. Experiments on DeepLabV3 and PSPNet across Pascal VOC and Cityscapes show stronger attack performance than baselines and improved robustness under AT, offering a new avenue for defenses in dense prediction tasks.

Abstract

Existing segmentation models exhibit significant vulnerability to adversarial attacks.To improve robustness, adversarial training incorporates adversarial examples into model training. However, existing attack methods consider only global semantic information and ignore contextual semantic relationships within the samples, limiting the effectiveness of adversarial training. To address this issue, we propose EroSeg-AT, a vulnerability-aware adversarial training framework that leverages EroSeg to generate adversarial examples. EroSeg first selects sensitive pixels based on pixel-level confidence and then progressively propagates perturbations to higher-confidence pixels, effectively disrupting the semantic consistency of the samples. Experimental results show that, compared to existing methods, our approach significantly improves attack effectiveness and enhances model robustness under adversarial training.
Paper Structure (13 sections, 5 equations, 5 figures, 1 table)

This paper contains 13 sections, 5 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Semantic Segmentation Models
  4. Adversarial Training
  5. Methodology
  6. Motivation
  7. EroSeg: A Complete Illustration
  8. Experiments
  9. Experimental Setup
  10. Attack Performance
  11. Ablation Study
  12. Adversarial training
  13. Limitation and Conclusion

Figures (5)

  • Figure 1: Illustration of adversarial training
  • Figure 2: Subfigure (a) shows the attack performance when targeting correctly classified pixels with confidence 1, those with confidence less than 1 (*), or all pixels (All). Subfigure (b) shows the effect of attacking only background (B), foreground (F), or all pixels (All).
  • Figure 3: The framework of EroSeg
  • Figure 4: The mIoU (%) results of ablation study. Including the effects of different initial thresholds (a), growth factors (b), weight factors (c), and attack iterations (d) on EroSeg.
  • Figure 5: The mIoU (%) results of comparison study.