Transferable Adversarial Attacks on SAM and Its Downstream Models

TL;DR

This work tackles the practical risk of adversarially attacking SAM-based downstream models without access to downstream data by introducing UMI-GRAT, which combines offline universal meta-initialization of perturbations with gradient-robust updates. The authors formalize the gradient-update deviation between the open-source SAM surrogate and fine-tuned downstream models and propose a gradient robust loss to mitigate it, enabling stronger transferability across unknown tasks. The two-stage approach—offline UMI learning and online GRAT adaptation—achieves state-of-the-art transfer performance on medical, shadow, and camouflage segmentation tasks and demonstrates plug-and-play compatibility with existing attacks. The results underscore significant security concerns for fine-tuned foundation-model deployments and motivate future defenses against such transferable adversarial threats.

Abstract

The utilization of large foundational models has a dilemma: while fine-tuning downstream tasks from them holds promise for making use of the well-generalized knowledge in practical applications, their open accessibility also poses threats of adverse usage. This paper, for the first time, explores the feasibility of adversarial attacking various downstream models fine-tuned from the segment anything model (SAM), by solely utilizing the information from the open-sourced SAM. In contrast to prevailing transfer-based adversarial attacks, we demonstrate the existence of adversarial dangers even without accessing the downstream task and dataset to train a similar surrogate model. To enhance the effectiveness of the adversarial attack towards models fine-tuned on unknown datasets, we propose a universal meta-initialization (UMI) algorithm to extract the intrinsic vulnerability inherent in the foundation model, which is then utilized as the prior knowledge to guide the generation of adversarial perturbations. Moreover, by formulating the gradient difference in the attacking process between the open-sourced SAM and its fine-tuned downstream models, we theoretically demonstrate that a deviation occurs in the adversarial update direction by directly maximizing the distance of encoded feature embeddings in the open-sourced SAM. Consequently, we propose a gradient robust loss that simulates the associated uncertainty with gradient-based noise augmentation to enhance the robustness of generated adversarial examples (AEs) towards this deviation, thus improving the transferability. Extensive experiments demonstrate the effectiveness of the proposed universal meta-initialized and gradient robust adversarial attack (UMI-GRAT) toward SAMs and their downstream models. Code is available at https://github.com/xiasong0501/GRAT.

Figures (6)

• Figure 1: An illustration of UMI-GRAT towards SAM and its downstream tasks. The UMI-GRAT can mislead various downstream models by solely utilizing information from the open-sourced SAM.
• Figure 2: The data flow of our UMI-GRAT, consisting of an offline learning process of UMI and a real-time gradient robust adversarial attack.
• Figure 3: The cosine similarity of white-box generated perturbations on surrogate and victim models.
• Figure 4: The $l_2$ distance of feature embedding from clean inputs and adversarial examples. The small distance gap between the surrogate and victim models indicates better transferability.
• Figure A.5: The visualized adversarial attack results in camouflaged object segmentation task.

Theorems & Definitions (3)

• Definition 1: Transferable adversarial attack via open-sourced SAM
• Definition 2: Universal and meta-initialized perturbation $\bm{\delta}$
• Proposition 1: Deviation in updating adversarial perturbation