Benchmarking Transferable Adversarial Attacks

TL;DR

This work provides the first systematic survey and benchmark of transferable adversarial attacks (TAA), categorizing methods into Generative Architecture, Semantic Similarity, Gradient Editing, Target Modification, and Ensemble approaches. It introduces TAA‑Bench, an extensible framework that reproduces ten representative TAA methods within a modular PyTorch pipeline and YAML‑driven configuration to enable fair cross‑model comparisons. The paper analyzes the strengths and limitations of each method across diverse architectures, highlighting practical trade‑offs between computation, complexity, and transferability. By delivering a reproducible benchmark and open‑source tools, it lays a foundation for advancing understanding of adversarial transferability and guiding defenses in real‑world settings.

Abstract

The robustness of deep learning models against adversarial attacks remains a pivotal concern. This study presents, for the first time, an exhaustive review of the transferability aspect of adversarial attacks. It systematically categorizes and critically evaluates various methodologies developed to augment the transferability of adversarial attacks. This study encompasses a spectrum of techniques, including Generative Structure, Semantic Similarity, Gradient Editing, Target Modification, and Ensemble Approach. Concurrently, this paper introduces a benchmark framework \textit{TAA-Bench}, integrating ten leading methodologies for adversarial attack transferability, thereby providing a standardized and systematic platform for comparative analysis across diverse model architectures. Through comprehensive scrutiny, we delineate the efficacy and constraints of each method, shedding light on their underlying operational principles and practical utility. This review endeavors to be a quintessential resource for both scholars and practitioners in the field, charting the complex terrain of adversarial transferability and setting a foundation for future explorations in this vital sector. The associated codebase is accessible at: https://github.com/KxPlaug/TAA-Bench