Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BadScan: An Architectural Backdoor Attack on Visual State Space Models

Om Suhas Deshmukh, Sankalp Nagaonkar, Achyut Mani Tripathi, Ashish Mishra

TL;DR

It is demonstrated that the BadScan backdoor attack represents a significant threat to visual state space models and remains effective even after complete retraining from scratch, achieving a higher Triggered Accuracy Ratio (TAR) in misleading the VMamba model and its variants.

Abstract

The newly introduced Visual State Space Model (VMamba), which employs \textit{State Space Mechanisms} (SSM) to interpret images as sequences of patches, has shown exceptional performance compared to Vision Transformers (ViT) across various computer vision tasks. However, recent studies have highlighted that deep models are susceptible to adversarial attacks. One common approach is to embed a trigger in the training data to retrain the model, causing it to misclassify data samples into a target class, a phenomenon known as a backdoor attack. In this paper, we first evaluate the robustness of the VMamba model against existing backdoor attacks. Based on this evaluation, we introduce a novel architectural backdoor attack, termed BadScan, designed to deceive the VMamba model. This attack utilizes bit plane slicing to create visually imperceptible backdoored images. During testing, if a trigger is detected by performing XOR operations between the $k^{th}$ bit planes of the modified triggered patches, the traditional 2D selective scan (SS2D) mechanism in the visual state space (VSS) block of VMamba is replaced with our newly designed BadScan block, which incorporates four newly developed scanning patterns. We demonstrate that the BadScan backdoor attack represents a significant threat to visual state space models and remains effective even after complete retraining from scratch. Experimental results on two widely used image classification datasets, CIFAR-10, and ImageNet-1K, reveal that while visual state space models generally exhibit robustness against current backdoor attacks, the BadScan attack is particularly effective, achieving a higher Triggered Accuracy Ratio (TAR) in misleading the VMamba model and its variants.

BadScan: An Architectural Backdoor Attack on Visual State Space Models

TL;DR

It is demonstrated that the BadScan backdoor attack represents a significant threat to visual state space models and remains effective even after complete retraining from scratch, achieving a higher Triggered Accuracy Ratio (TAR) in misleading the VMamba model and its variants.

Abstract

The newly introduced Visual State Space Model (VMamba), which employs \textit{State Space Mechanisms} (SSM) to interpret images as sequences of patches, has shown exceptional performance compared to Vision Transformers (ViT) across various computer vision tasks. However, recent studies have highlighted that deep models are susceptible to adversarial attacks. One common approach is to embed a trigger in the training data to retrain the model, causing it to misclassify data samples into a target class, a phenomenon known as a backdoor attack. In this paper, we first evaluate the robustness of the VMamba model against existing backdoor attacks. Based on this evaluation, we introduce a novel architectural backdoor attack, termed BadScan, designed to deceive the VMamba model. This attack utilizes bit plane slicing to create visually imperceptible backdoored images. During testing, if a trigger is detected by performing XOR operations between the bit planes of the modified triggered patches, the traditional 2D selective scan (SS2D) mechanism in the visual state space (VSS) block of VMamba is replaced with our newly designed BadScan block, which incorporates four newly developed scanning patterns. We demonstrate that the BadScan backdoor attack represents a significant threat to visual state space models and remains effective even after complete retraining from scratch. Experimental results on two widely used image classification datasets, CIFAR-10, and ImageNet-1K, reveal that while visual state space models generally exhibit robustness against current backdoor attacks, the BadScan attack is particularly effective, achieving a higher Triggered Accuracy Ratio (TAR) in misleading the VMamba model and its variants.

Paper Structure

This paper contains 26 sections, 8 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. SSM in Vision
  4. Backdoor Attacks on VSS Models
  5. Types of Backdoor Attacks
  6. Defense Against Backdoor Attacks
  7. Methodology
  8. State Space Model
  9. BadScan Attack
  10. Generation, Insertion and Detection of a Trigger
  11. BadScan Block
  12. Experiments & Results
  13. Image Datasets
  14. Deep Networks
  15. Experimental Details
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Workflow of Trigger Generation and Detection (Illustrated with a 3-Bit Representation for Better Clarity)
  • Figure 2: SS2D-Based VSS Block (Left) zhu2024vision Vs BadScan Block (Right)
  • Figure 3: Working of RES, REAS, REMS and REDS Scanning Patterns Inside BadScan Block
  • Figure 4: Clean and Attacked Images from CIFAR-10 (Target Class= Ships, Source Class= Deer)