Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidence

Hanbin Hong, Xinyu Zhang, Binghui Wang, Zhongjie Ba, Yuan Hong

TL;DR

This work establishes a novel theoretical foundation for ensuring the ASP of the black-box attack with randomized adversarial examples (AEs) and proposes several novel techniques to craft the randomized AEs while reducing the perturbation size for better imperceptibility.

Abstract

Black-box adversarial attacks have demonstrated strong potential to compromise machine learning models by iteratively querying the target model or leveraging transferability from a local surrogate model. Recently, such attacks can be effectively mitigated by state-of-the-art (SOTA) defenses, e.g., detection via the pattern of sequential queries, or injecting noise into the model. To our best knowledge, we take the first step to study a new paradigm of black-box attacks with provable guarantees -- certifiable black-box attacks that can guarantee the attack success probability (ASP) of adversarial examples before querying over the target model. This new black-box attack unveils significant vulnerabilities of machine learning models, compared to traditional empirical black-box attacks, e.g., breaking strong SOTA defenses with provable confidence, constructing a space of (infinite) adversarial examples with high ASP, and the ASP of the generated adversarial examples is theoretically guaranteed without verification/queries over the target model. Specifically, we establish a novel theoretical foundation for ensuring the ASP of the black-box attack with randomized adversarial examples (AEs). Then, we propose several novel techniques to craft the randomized AEs while reducing the perturbation size for better imperceptibility. Finally, we have comprehensively evaluated the certifiable black-box attacks on the CIFAR10/100, ImageNet, and LibriSpeech datasets, while benchmarking with 16 SOTA black-box attacks, against various SOTA defenses in the domains of computer vision and speech recognition. Both theoretical and experimental results have validated the significance of the proposed attack. The code and all the benchmarks are available at \url{https://github.com/datasec-lab/CertifiedAttack}.

Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidence

TL;DR

This work establishes a novel theoretical foundation for ensuring the ASP of the black-box attack with randomized adversarial examples (AEs) and proposes several novel techniques to craft the randomized AEs while reducing the perturbation size for better imperceptibility.

Abstract

Black-box adversarial attacks have demonstrated strong potential to compromise machine learning models by iteratively querying the target model or leveraging transferability from a local surrogate model. Recently, such attacks can be effectively mitigated by state-of-the-art (SOTA) defenses, e.g., detection via the pattern of sequential queries, or injecting noise into the model. To our best knowledge, we take the first step to study a new paradigm of black-box attacks with provable guarantees -- certifiable black-box attacks that can guarantee the attack success probability (ASP) of adversarial examples before querying over the target model. This new black-box attack unveils significant vulnerabilities of machine learning models, compared to traditional empirical black-box attacks, e.g., breaking strong SOTA defenses with provable confidence, constructing a space of (infinite) adversarial examples with high ASP, and the ASP of the generated adversarial examples is theoretically guaranteed without verification/queries over the target model. Specifically, we establish a novel theoretical foundation for ensuring the ASP of the black-box attack with randomized adversarial examples (AEs). Then, we propose several novel techniques to craft the randomized AEs while reducing the perturbation size for better imperceptibility. Finally, we have comprehensively evaluated the certifiable black-box attacks on the CIFAR10/100, ImageNet, and LibriSpeech datasets, while benchmarking with 16 SOTA black-box attacks, against various SOTA defenses in the domains of computer vision and speech recognition. Both theoretical and experimental results have validated the significance of the proposed attack. The code and all the benchmarks are available at \url{https://github.com/datasec-lab/CertifiedAttack}.
Paper Structure (45 sections, 5 theorems, 28 equations, 9 figures, 41 tables, 7 algorithms)

This paper contains 45 sections, 5 theorems, 28 equations, 9 figures, 41 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Certifiable Attacks vs. Empirical Attacks
  3. Randomization for Certifiable Attacks
  4. Problem Definition
  5. Attack Overview
  6. Certifiable Black-box Attack
  7. Adversarial Distribution Localization
  8. Randomized Parallel Query
  9. Proposed Localization Algorithms
  10. Adversarial Distribution Refinement
  11. Certification for Adversarial Distribution Shifting
  12. Obtaining Refined Adversarial Distribution
  13. Discussions on Our Attack
  14. Evaluations
  15. Experimental Setup
  16. ...and 30 more sections

Key Result

Theorem 1

(Certifiable Adversarial Distribution Shifting) Let $f$ be a classifier, $\epsilon$ be the noise drawn from any continuous probability density function $\varphi(0,\boldsymbol{\kappa})$. Let $p$ be the predefined attack success possibility threshold. Denote $\underline{p_{adv}}$ as the lower bound of $\mathbb{P}[f(x'+\delta+\epsilon)\neq y]\geq p$ is guaranteed for any shifting vector $\delta$ when

Figures (9)

  • Figure 1: Empirical attacks vs. Certifiable attacks. (a) Certifiable attack can break the SOTA AE detection and randomized defenses. (b) Certifiable attack uncovers space-wise vulnerability rather than sample-wise vulnerability. (c) Once certified, Certifiable Attack can generate unlimited unique AEs with a guaranteed minimum ASP without querying the model for verification, while the empirical attack requires verifying the attack result of crafted AE by query.
  • Figure 2: Overview of our certifiable black-box attack to generate certified adversarial distribution.
  • Figure 3: Illustration of randomized parallel query (returning the probability $Q(x')$ that $x'+\epsilon$ is an adversarial example).
  • Figure 4: Illustration of geometrically shifting.
  • Figure 5: t-SNE visualization of adversarial example sampling from the adversarial distribution.
  • ...and 4 more figures

Theorems & Definitions (6)

  • definition 1: Certifiable black-box attack
  • Theorem 1
  • Theorem 2
  • Lemma 1
  • Corollary 2.1
  • Lemma 2