Effects of Exponential Gaussian Distribution on (Double Sampling) Randomized Smoothing

TL;DR

The paper investigates how two distribution families, ESG and EGG, interact with Randomized Smoothing (RS) and Double Sampling RS (DSRS). It derives an analytic ESG-certified radius formula that aligns with Gaussian-based certs in high dimensions and shows ESG is largely agnostic to the exponent $\eta$, while introducing EGG and demonstrating potential DSRS improvements via concentration-based analysis, yielding tighter $\ell_2$ lower bounds when $\eta\in(0,2]$. Under a concentration framework, EGG can achieve $\Omega(d^{1/\eta})$ bounds (and hence $\Omega(\sqrt{d})$ for small $\eta$), suggesting a path to mitigating the curse of dimensionality in RS, with experiments reporting up to 6.4% gain in certified accuracy on ImageNet. Overall, ESG extends the set of smoothing distributions without sacrificing certification, and EGG provides a mechanism to enhance DSRS performance in high dimensions, contingent on classifier concentration properties.

Abstract

Randomized Smoothing (RS) is currently a scalable certified defense method providing robustness certification against adversarial examples. Although significant progress has been achieved in providing defenses against $\ell_p$ adversaries, the interaction between the smoothing distribution and the robustness certification still remains vague. In this work, we comprehensively study the effect of two families of distributions, named Exponential Standard Gaussian (ESG) and Exponential General Gaussian (EGG) distributions, on Randomized Smoothing and Double Sampling Randomized Smoothing (DSRS). We derive an analytic formula for ESG's certified radius, which converges to the origin formula of RS as the dimension $d$ increases. Additionally, we prove that EGG can provide tighter constant factors than DSRS in providing $Ω(\sqrt{d})$ lower bounds of $\ell_2$ certified radius, and thus further addresses the curse of dimensionality in RS. Our experiments on real-world datasets confirm our theoretical analysis of the ESG distributions, that they provide almost the same certification under different exponents $η$ for both RS and DSRS. In addition, EGG brings a significant improvement to the DSRS certification, but the mechanism can be different when the classifier properties are different. Compared to the primitive DSRS, the increase in certified accuracy provided by EGG is prominent, up to 6.4% on ImageNet.

Figures (17)

• Figure 1: Our analytic formula for ESG highly approximates cohen2019's at a sufficiently large dimension. (Both $\sigma=1.0$. Left: $d=3072$, Right: $d=150224$.)
• Figure 2: Numerical simulations for EGG in DSRS. Left: the concentration property holds ($B=1$), smaller $\eta$ provides tighter lower bounds. Right: the concentration property does not hold ($B<1$), larger $\eta$ provides better certified radius. For definitions of $A$ and $B$, see Equation (\ref{['primaldsrs']}).
• Figure 3: Tight factor $\mu$ grows as $\eta$ shrinks, for most $d-2k \in [1, 30] \cap \mathbb{N}$.
• Figure 4: Illustration for experiments.
• Figure 5: ACR results on real-world datasets. (a). ACR monotonically increases with $\eta$ in EGG. (b). The ACR growth gain from DSRS relative to NP shrinks with $\eta$ in EGG. (c). ACR stays almost constant in ESG. (d). The ACR growth gain from DSRS remains almost constant in ESG. For (a) and (c), solid lines represent results from DSRS, and dotted lines represent results from NP.

Theorems & Definitions (28)

• Definition 3.1
• Theorem 5.1
• Theorem 5.2
• Lemma 5.4
• Lemma 5.6
• Definition 6.1
• Theorem 6.2: EGG with $\eta \in(0,2)$ can certify $\Omega(\sqrt{d})$ lower bounds
• Lemma 2.1
• proof
• Theorem 2.2