Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Batch Normalization Networks for Adversarial Robustness

Shao-Yuan Lo, Vishal M. Patel

TL;DR

This paper proposes a novel defense, referred to as the Adaptive Batch Normalization Network (ABNN), which employs a pre-trained substitute model to generate clean BN statistics and sends them to the target model, which is exclusively trained on clean data.

Abstract

Deep networks are vulnerable to adversarial examples. Adversarial Training (AT) has been a standard foundation of modern adversarial defense approaches due to its remarkable effectiveness. However, AT is extremely time-consuming, refraining it from wide deployment in practical applications. In this paper, we aim at a non-AT defense: How to design a defense method that gets rid of AT but is still robust against strong adversarial attacks? To answer this question, we resort to adaptive Batch Normalization (BN), inspired by the recent advances in test-time domain adaptation. We propose a novel defense accordingly, referred to as the Adaptive Batch Normalization Network (ABNN). ABNN employs a pre-trained substitute model to generate clean BN statistics and sends them to the target model. The target model is exclusively trained on clean data and learns to align the substitute model's BN statistics. Experimental results show that ABNN consistently improves adversarial robustness against both digital and physically realizable attacks on both image and video datasets. Furthermore, ABNN can achieve higher clean data performance and significantly lower training time complexity compared to AT-based approaches.

Adaptive Batch Normalization Networks for Adversarial Robustness

TL;DR

This paper proposes a novel defense, referred to as the Adaptive Batch Normalization Network (ABNN), which employs a pre-trained substitute model to generate clean BN statistics and sends them to the target model, which is exclusively trained on clean data.

Abstract

Deep networks are vulnerable to adversarial examples. Adversarial Training (AT) has been a standard foundation of modern adversarial defense approaches due to its remarkable effectiveness. However, AT is extremely time-consuming, refraining it from wide deployment in practical applications. In this paper, we aim at a non-AT defense: How to design a defense method that gets rid of AT but is still robust against strong adversarial attacks? To answer this question, we resort to adaptive Batch Normalization (BN), inspired by the recent advances in test-time domain adaptation. We propose a novel defense accordingly, referred to as the Adaptive Batch Normalization Network (ABNN). ABNN employs a pre-trained substitute model to generate clean BN statistics and sends them to the target model. The target model is exclusively trained on clean data and learns to align the substitute model's BN statistics. Experimental results show that ABNN consistently improves adversarial robustness against both digital and physically realizable attacks on both image and video datasets. Furthermore, ABNN can achieve higher clean data performance and significantly lower training time complexity compared to AT-based approaches.
Paper Structure (10 sections, 6 equations, 3 figures, 3 tables)

This paper contains 10 sections, 6 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Adaptive Batch Normalization Layer
  4. Training and Inference
  5. Discussion
  6. Experiments
  7. Experimental Setup
  8. Evaluation Results
  9. Training Time Complexity
  10. Conclusion

Figures (3)

  • Figure 1: Summary of ABNN's main strengths: (a) Better clean data performance, (b) better robustness generalization, and (c) better training efficiency. Detailed analysis is presented in Section \ref{['sec:3']}.
  • Figure 2: The proposed ABNN framework. ABNN employs a pre-trained and frozen substitute model to generate cleaner BN statistics, and sends them to the target model. The target model is exclusively trained on clean data and learns to align the substitute model's BN statistics.
  • Figure 3: Examples of clean data, the PGD attack, and the ROA attack. The PGD attack is typically imperceptible to human eyes. The PGD perturbations are $15 \times$ magnified for better visualization. The ROA attack is like pasting a rectangular adversarial sticker on the input.