Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Certifying Global Robustness for Deep Neural Networks

You Li, Guannan Zhao, Shuyu Kong, Yunqi He, Hai Zhou

TL;DR

This work tackles the challenge of certifying global robustness for deep neural networks by shifting from local neighborhood verification to probabilistic modeling of meaningful input regions. It combines PAC robustness verification with probabilistic programs to define a global input distribution and introduces the cumulative robustness function as a comprehensive robustness metric. The ACE framework couples local robustness estimation with adaptive multi-level splitting and regression to efficiently and accurately quantify global robustness, even for rare violations, while enabling mining of diverse counterexamples for adversarial training. The results demonstrate that ACE yields faster, more accurate global robustness estimates and practical counterexamples, offering scalable, rigorous guarantees with meaningful real-world input distributions.

Abstract

A globally robust deep neural network resists perturbations on all meaningful inputs. Current robustness certification methods emphasize local robustness, struggling to scale and generalize. This paper presents a systematic and efficient method to evaluate and verify global robustness for deep neural networks, leveraging the PAC verification framework for solid guarantees on verification results. We utilize probabilistic programs to characterize meaningful input regions, setting a realistic standard for global robustness. Additionally, we introduce the cumulative robustness curve as a criterion in evaluating global robustness. We design a statistical method that combines multi-level splitting and regression analysis for the estimation, significantly reducing the execution time. Experimental results demonstrate the efficiency and effectiveness of our verification method and its capability to find rare and diversified counterexamples for adversarial training.

Certifying Global Robustness for Deep Neural Networks

TL;DR

This work tackles the challenge of certifying global robustness for deep neural networks by shifting from local neighborhood verification to probabilistic modeling of meaningful input regions. It combines PAC robustness verification with probabilistic programs to define a global input distribution and introduces the cumulative robustness function as a comprehensive robustness metric. The ACE framework couples local robustness estimation with adaptive multi-level splitting and regression to efficiently and accurately quantify global robustness, even for rare violations, while enabling mining of diverse counterexamples for adversarial training. The results demonstrate that ACE yields faster, more accurate global robustness estimates and practical counterexamples, offering scalable, rigorous guarantees with meaningful real-world input distributions.

Abstract

A globally robust deep neural network resists perturbations on all meaningful inputs. Current robustness certification methods emphasize local robustness, struggling to scale and generalize. This paper presents a systematic and efficient method to evaluate and verify global robustness for deep neural networks, leveraging the PAC verification framework for solid guarantees on verification results. We utilize probabilistic programs to characterize meaningful input regions, setting a realistic standard for global robustness. Additionally, we introduce the cumulative robustness curve as a criterion in evaluating global robustness. We design a statistical method that combines multi-level splitting and regression analysis for the estimation, significantly reducing the execution time. Experimental results demonstrate the efficiency and effectiveness of our verification method and its capability to find rare and diversified counterexamples for adversarial training.
Paper Structure (25 sections, 1 theorem, 11 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 25 sections, 1 theorem, 11 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Global Robustness of Deep Neural Networks
  3. Preliminaries
  4. Analysis of Global Robustness
  5. Characterizing the Global Input Distribution
  6. Statistical Global Robustness Verification
  7. PAC Robustness Verification
  8. Global Robustness Certification and Measurement
  9. Global Robustness Criterion.
  10. Cumulative Robustness Function.
  11. Parameter Estimation of Local Robustness Risk.
  12. Adaptive Multi-level Splitting.
  13. The Global Robustness Certification Algorithm
  14. Evaluation
  15. General Performance
  16. ...and 10 more sections

Key Result

Proposition 1

Let $x$ be an input sample and $x'$ be drawn from $\mathbb{B}(x,r)$ uniformly at random. If the probability distribution of $h(x, x', m)$ follows a normal distribution $\mathcal{N}(\mu_{h},\sigma_{h}^2)$, the local robustness risk $\mathbf{E}_{x' \sim \mathbb{B}(x,r)} \mathbbm{1}[m(f_{\theta}, x, x'

Figures (3)

  • Figure 1: Assessment of parameter estimation with (solid line) and without (dashed line) regression. Local robustness risks (p) of the points are computed by AMLS.
  • Figure 2: The cumulative robustness functions for classes 1-5 (a) and 6-10 (b).
  • Figure 3: Counterexamples mis-classified to designated classes.

Theorems & Definitions (3)

  • definition thmcounterdefinition: Local Robustness
  • Proposition 1
  • proof