Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defenses

Chong Xiang, Tong Wu, Sihui Dai, Jonathan Petit, Suman Jana, Prateek Mittal

TL;DR

PatchCURE addresses adversarial patch defenses by tackling the three-way trade-off among certifiable robustness, model utility, and computation efficiency. It unifies small-receptive-field (SRF) and large-receptive-field (LRF) approaches into a three-module framework—an SRF sub-model, an LRF sub-model, and a secure operation—with a tunable split point $k$ to navigate the trade-offs. The method provides certifiable guarantees via PCURE-Certify by mapping image-space attacks to feature-space threat models and applying a robust prediction procedure; efficient instantiations achieve throughput near undefended models while delivering state-of-the-art clean and robust accuracy across efficiency levels. PatchCURE demonstrates viability on ImageNet-1k with ViT-SRF and BagNet SRF options and offers a flexible path to adapt defenses to practical constraints and evolving SRF/LRF advances. Overall, PatchCURE presents a versatile, deployable framework for certifiably robust defenses that can interpolate between efficiency and robustness as needed.

Abstract

State-of-the-art defenses against adversarial patch attacks can now achieve strong certifiable robustness with a marginal drop in model utility. However, this impressive performance typically comes at the cost of 10-100x more inference-time computation compared to undefended models -- the research community has witnessed an intense three-way trade-off between certifiable robustness, model utility, and computation efficiency. In this paper, we propose a defense framework named PatchCURE to approach this trade-off problem. PatchCURE provides sufficient "knobs" for tuning defense performance and allows us to build a family of defenses: the most robust PatchCURE instance can match the performance of any existing state-of-the-art defense (without efficiency considerations); the most efficient PatchCURE instance has similar inference efficiency as undefended models. Notably, PatchCURE achieves state-of-the-art robustness and utility performance across all different efficiency levels, e.g., 16-23% absolute clean accuracy and certified robust accuracy advantages over prior defenses when requiring computation efficiency to be close to undefended models. The family of PatchCURE defenses enables us to flexibly choose appropriate defenses to satisfy given computation and/or utility constraints in practice.

PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defenses

TL;DR

PatchCURE addresses adversarial patch defenses by tackling the three-way trade-off among certifiable robustness, model utility, and computation efficiency. It unifies small-receptive-field (SRF) and large-receptive-field (LRF) approaches into a three-module framework—an SRF sub-model, an LRF sub-model, and a secure operation—with a tunable split point to navigate the trade-offs. The method provides certifiable guarantees via PCURE-Certify by mapping image-space attacks to feature-space threat models and applying a robust prediction procedure; efficient instantiations achieve throughput near undefended models while delivering state-of-the-art clean and robust accuracy across efficiency levels. PatchCURE demonstrates viability on ImageNet-1k with ViT-SRF and BagNet SRF options and offers a flexible path to adapt defenses to practical constraints and evolving SRF/LRF advances. Overall, PatchCURE presents a versatile, deployable framework for certifiably robust defenses that can interpolate between efficiency and robustness as needed.

Abstract

State-of-the-art defenses against adversarial patch attacks can now achieve strong certifiable robustness with a marginal drop in model utility. However, this impressive performance typically comes at the cost of 10-100x more inference-time computation compared to undefended models -- the research community has witnessed an intense three-way trade-off between certifiable robustness, model utility, and computation efficiency. In this paper, we propose a defense framework named PatchCURE to approach this trade-off problem. PatchCURE provides sufficient "knobs" for tuning defense performance and allows us to build a family of defenses: the most robust PatchCURE instance can match the performance of any existing state-of-the-art defense (without efficiency considerations); the most efficient PatchCURE instance has similar inference efficiency as undefended models. Notably, PatchCURE achieves state-of-the-art robustness and utility performance across all different efficiency levels, e.g., 16-23% absolute clean accuracy and certified robust accuracy advantages over prior defenses when requiring computation efficiency to be close to undefended models. The family of PatchCURE defenses enables us to flexibly choose appropriate defenses to satisfy given computation and/or utility constraints in practice.
Paper Structure (25 sections, 3 theorems, 5 equations, 9 figures, 13 tables, 3 algorithms)

This paper contains 25 sections, 3 theorems, 5 equations, 9 figures, 13 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. Receptive Fields of Vision Models
  5. Overview of SRF and LRF Defenses
  6. PatchCURE Framework
  7. PatchCURE Insights
  8. PatchCURE Algorithm
  9. Building SRF Models
  10. Robustness Certification
  11. PatchCURE Instantiation
  12. Evaluation
  13. Setup
  14. Main results
  15. Detailed Analyses
  16. ...and 10 more sections

Key Result

Proposition 1

Given a correctly implemented $\textsc{Map}(\cdot)$, an image-space threat model ${\mathcal{A}}_{\mathcal{R}}$, an SRF sub-model ${\mathbb M}_\text{srf}$, an input image $\mathbf{x}$, and the converted feature-space threat model ${\mathcal{A}}_{\mathcal{R}}^f=\textsc{Map}({\mathbb M}_\text{srf},{\ma

Figures (9)

  • Figure 1: PatchCURE overview.I. PatchCURE inference (Section \ref{['sec-defense-alg']}): Given an input image, we first call an SRF (small receptive field) sub-model once to extract an intermediate feature map. The use of SRF ensures that only part of the features is corrupted. Next, we leverage secure operation, which typically involves multiple calls to an LRF (large receptive field) sub-model, for final predictions. II. PatchCURE for the trade-off problem (Section \ref{['sec-defense-discussion']}). We can adjust the combination of SRF and LRF layers to balance the three-way trade-off. As we use fewer SRF layers and more LRF layers, the defense model (with a fixed number of total layers) normally has larger receptive fields, larger model capacity, better model utility, and higher certifiable robustness, but poorer computation efficiency.
  • Figure 2: Certified robust accuracy and inference throughput (img/s) for different defenses on ImageNet-1k imagenet: (i) our PatchCURE instances with different settings; (ii) PatchCleanser xiang2022patchcleanser -- also a special instance of PatchCURE; (iii) DRS+ViT, including Smoothed ViT salman2022certified, ECViT ecvit, and ViP li2022vip; (iv) PatchGuard xiang2021patchguard; (v) BagCert bagcert; (vi) De-Randomized Smoothing (DRS) levine2020randomized; (vii) Clipped BagNet (CBN) zhang2020clipped; (viii) undefended ViT vit. Certified robustness considers one 2%-pixel square patch anywhere on the image.
  • Figure 3: Illustration of model receptive field. For a convolutional network with a kernel size of $3$ and stride size of $1$, the blue cell in Layer $2$ is affected by $3\times3$ cells in Layer $1$ and $5\times5$ cells in Layer $0$ (the model input).
  • Figure 4: Local attention: each square is a visual token.
  • Figure 5: Effect of the splitting layer $k$ on ViT14x2-based PatchCURE
  • ...and 4 more figures

Theorems & Definitions (5)

  • Proposition 1: Correctness of $\textsc{Map}(\cdot)$
  • Proposition 2: Correctness of $\textsc{SO-Cert}(\cdot)$
  • Theorem 1
  • proof
  • Definition 1: $\mathcal{R}$-covering