Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustness

TL;DR

The paper tackles the high computational cost of certifying robustness in randomized smoothing by reframing the estimation problem as adaptive, sequential statistical inference. It introduces a strictly improved randomized Clopper-Pearson interval and, more importantly, confidence sequences to enable data-driven stopping and reduced sample complexity in certification tasks. The authors provide theoretical lower and upper bounds on the width of these intervals and sequences, show near-optimal adaptive performance, and validate the approach empirically on robustness tasks. This work enables faster, practically feasible certified robustness by replacing fixed-sample procedures with adaptive, time-uniform guarantees that balance type-1 and type-2 errors with sample availability. Overall, it broadens the applicability of randomized smoothing by making certification more efficient across modalities and threat models.

Abstract

Randomized smoothing is a popular certified defense against adversarial attacks. In its essence, we need to solve a problem of statistical estimation which is usually very time-consuming since we need to perform numerous (usually $10^5$) forward passes of the classifier for every point to be certified. In this paper, we review the statistical estimation problems for randomized smoothing to find out if the computational burden is necessary. In particular, we consider the (standard) task of adversarial robustness where we need to decide if a point is robust at a certain radius or not using as few samples as possible while maintaining statistical guarantees. We present estimation procedures employing confidence sequences enjoying the same statistical guarantees as the standard methods, with the optimal sample complexities for the estimation task and empirically demonstrate their good performance. Additionally, we provide a randomized version of Clopper-Pearson confidence intervals resulting in strictly stronger certificates.

Figures (5)

• Figure 1: left: Comparison of coverages of confidence intervals for the mean estimation of $\mathcal{B}(100, p)$ when $\alpha := 0.001$. Note that for $p > \sqrt[n]{\alpha} \sim 0.93$, the coverage is $1$. right: Comparison of $\ell_2$-robustness curves with the standard (dashed) or the randomized (solid) Clopper-Pearson bounds on a CIFAR-10 dataset under the standard setting. The experimental details are in Appendix \ref{['exp:details']}.
• Figure 2: left: Comparison of widths of confidence sequences for the mean of Bernoulli $\mathcal{B}(0.1)$ with $\alpha=0.001$. The width is on top and the actual confidence sequence on the bottom. In the notation of Algorithms \ref{['alg:union_bound']} and \ref{['alg:sprt_conf_seq']}, the sequence of $U-L$ is in the top figure, while both sequences $U$ and $L$ are in the bottom figure. Note the $\log$-scale for $t$ (and width on top). $\textbf{right}:$ Instantiation of Task \ref{['def:seq_dec_task']}. The goal is to decide if $p = 0.91$ (vertical magenta line) or not with $\alpha = 0.001$. On top are the numbers of samples requested for the individual methods averaged over $1000$ trials for $51$ equally spaced values of $p\in[0,1]$; on the bottom is the relative suboptimality of the individual methods; i.e., how many times more samples did they request compared to the ideal method. Note log scales on the $y-$axis. methods: UBnd-CS and Betting-CS are from Algorithm \ref{['alg:union_bound']} and \ref{['alg:sprt_conf_seq']} respectively. Adaptive is from horvath2021boosting. The ideal is the unattainable lower-bound for the two tasks. On the LHS, it is a confidence interval on level $1-\alpha$ computed independently at every time step. On the RHS, it is SPRT knowing both $p,q$ which is optimal due to wald2004sequential.
• Figure 3: Actual coverages for (randomized) Clopper-Pearson confidence intervals for $\mathcal{B}(2,p)$.
• Figure 4: Comparison of the robustness curves for binary and multiclass certification. In the binary case, all the failure budget $\alpha=0.001$ was spent on controlling the top-1 class probability. In the multiclass setting, we spend $\lambda$ fraction of the budget in bounding $p_A$ and the remaining $1-\lambda$ part on bounding $p_B$. Note that this has no significant effect. The average certified radius for binary certification is $0.50$, while for the multiclass it is $0.61$. The experimental details are in Appendix \ref{['exp:details']}. The only difference is that now $\sigma=1$.
• Figure 5: Samples needed for the adaptive estimation task as in Figure \ref{['fig:comp_cs']} for different hyperparameters. $\beta$ is the factor by which we enlarge the sample size before computing new confidence interval, $\gamma$ is the scaling of $\alpha$ as described in the main text. I.e., $k-$th estimation will have $\alpha_k = \frac{\alpha c}{k^\gamma}$ where $c$ is the normalization constant such that $\sum_{k=1}^\infty {\alpha_k} = \alpha$.

Theorems & Definitions (19)

• Definition 2.1: Confidence interval for binomials
• Definition 2.2: Clopper-Pearson intervals
• Definition 2.3: Randomized Clopper-Pearson intervals
• Proposition 2.4
• Definition 2.5: Confidence sequence
• Remark 2.6
• Theorem 2.7
• Corollary 2.8
• Definition 2.9: Martingale
• Proposition 2.10: Ville's inequality durrett2010probability