Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothing

TL;DR

This work addresses the decline of the ${\ell_2}$ certified radius in Randomized Smoothing as input dimension grows by introducing Dual Randomized Smoothing (DRS), which down-samples a high-dimensional input into two lower-dimensional sub-images and smooths them separately. Theoretical analysis yields a tight ${\ell_2}$ radius bound that scales with $(1/\sqrt{m} + 1/\sqrt{n})$ rather than $1/\sqrt{d}$, thereby mitigating the curse of dimensionality. Empirical results on CIFAR-10 and ImageNet show that DRS consistently improves certified accuracy and average certified radius, and it integrates with existing RS methods and ensemble boosting to further enhance robustness. The approach leverages spatial redundancy via simple down-sampling kernels and provides practical, code-enabled gains for certified robustness in vision models.

Abstract

Randomized Smoothing (RS) has been proven a promising method for endowing an arbitrary image classifier with certified robustness. However, the substantial uncertainty inherent in the high-dimensional isotropic Gaussian noise imposes the curse of dimensionality on RS. Specifically, the upper bound of ${\ell_2}$ certified robustness radius provided by RS exhibits a diminishing trend with the expansion of the input dimension $d$, proportionally decreasing at a rate of $1/\sqrt{d}$. This paper explores the feasibility of providing ${\ell_2}$ certified robustness for high-dimensional input through the utilization of dual smoothing in the lower-dimensional space. The proposed Dual Randomized Smoothing (DRS) down-samples the input image into two sub-images and smooths the two sub-images in lower dimensions. Theoretically, we prove that DRS guarantees a tight ${\ell_2}$ certified robustness radius for the original input and reveal that DRS attains a superior upper bound on the ${\ell_2}$ robustness radius, which decreases proportionally at a rate of $(1/\sqrt m + 1/\sqrt n )$ with $m+n=d$. Extensive experiments demonstrate the generalizability and effectiveness of DRS, which exhibits a notable capability to integrate with established methodologies, yielding substantial improvements in both accuracy and ${\ell_2}$ certified robustness baselines of RS on the CIFAR-10 and ImageNet datasets. Code is available at https://github.com/xiasong0501/DRS.

Figures (6)

• Figure 1: (a) The smoothing process of DRS. (b) The upper bound of ${\ell_2}$ certified radius (calculated by \ref{['eq:RS_radiusbound']} and \ref{['eq:DRS_uppperbound']}) of RS and DRS with $\sigma={1}/{\sqrt d }$ and smoothed probability $=0.999$.
• Figure 2: The implementation of Dual Randomized Smoothing (DRS). The image is down-sampled into two non-overlapping sub-images by utilizing two predefined 2x2 pixel indexes.
• Figure 3: The accuracy and robustness trade-off curve of RS and DRS on CIFAR-10 dataset. The data is collected by training multiple models using noise with $\sigma \in \left[ {0.07,0.7} \right]$. We fit this curve by a second-order polynomial function.
• Figure 4: (a) The visualized landscape of objective function for $k=2$ across various $\tilde{p}$. (b)The visualized landscape of objective function for $k=3$ with $\tilde{p}=1.50$.
• Figure 5: The visualized landscape of objective function and for smoothing with different variances. For symmetric cases where $\eta=2$ and $\eta=0.5$, the sum of the two optimal $\mathop{p'}\nolimits_A^l$ is $\tilde{p}$.

Theorems & Definitions (5)

• Theorem 1
• Proposition 1
• Theorem 2
• proof
• Lemma 1