Towards Strong Certified Defense with Universal Asymmetric Randomization

TL;DR

UCAN tackles the limitation of isotropic noise in randomized smoothing by introducing universally certifiable anisotropic noise. It provides a universal linear transformation and three Noise Parameter Generators (NPGs) to tailor per-dimension noise means and variances, enabling certification under arbitrary $\ell_p$ norms. Empirical results on MNIST, CIFAR10, and ImageNet show substantial gains in certified accuracy and radius, with up to $182.6\%$ improvement at large radii. This work broadens the practicality of provable robustness for diverse data domains by matching noise to input structure while preserving soundness.

Abstract

Randomized smoothing has become essential for achieving certified adversarial robustness in machine learning models. However, current methods primarily use isotropic noise distributions that are uniform across all data dimensions, such as image pixels, limiting the effectiveness of robustness certification by ignoring the heterogeneity of inputs and data dimensions. To address this limitation, we propose UCAN: a novel technique that \underline{U}niversally \underline{C}ertifies adversarial robustness with \underline{A}nisotropic \underline{N}oise. UCAN is designed to enhance any existing randomized smoothing method, transforming it from symmetric (isotropic) to asymmetric (anisotropic) noise distributions, thereby offering a more tailored defense against adversarial attacks. Our theoretical framework is versatile, supporting a wide array of noise distributions for certified robustness in different $\ell_p$-norms and applicable to any arbitrary classifier by guaranteeing the classifier's prediction over perturbed inputs with provable robustness bounds through tailored noise injection. Additionally, we develop a novel framework equipped with three exemplary noise parameter generators (NPGs) to optimally fine-tune the anisotropic noise parameters for different data dimensions, allowing for pursuing different levels of robustness enhancements in practice.Empirical evaluations underscore the significant leap in UCAN's performance over existing state-of-the-art methods, demonstrating up to $182.6\%$ improvement in certified accuracy at large certified radii on MNIST, CIFAR10, and ImageNet datasets.\footnote{Code is anonymously available at \href{https://github.com/youbin2014/UCAN/}{https://github.com/youbin2014/UCAN/}}

Figures (10)

• Figure 1: Anisotropic noise vs isotropic noise. The decision regions of $f$ are denoted in different colors. The dashed lines are the level sets of the noise distribution. The left figure shows the RS with isotropic Gaussian noise $\mathcal{N}(0, \lambda^2 \mathbf{I})$ in cohen2019certified whereas the right figure shows the RS with anisotropic Gaussian noise $\mathcal{N}(\mu, \Sigma)$, where $\Sigma=\lambda^2 diag (\sigma_1^2, \sigma_2^2, ..., \sigma_d^2)$. Certification can be improved by enlarging the gap between $\underline{p_A}$ and $\overline{p_B}$.
• Figure 2: An example illustration of the full certified region (green) and the certified radius (blue), where $\sigma_1 < \sigma_2$.
• Figure 3: Left: the framework for customizing anisotropic noise. Right: three example noise parameter generators (NPGs).
• Figure 4: Spatial distributions for noise variances (pattern-wise).
• Figure 5: Architecture of parameter generator for certification-wise anisotropic noise.

Theorems & Definitions (20)

• Theorem 1: Certified Robustness via Randomized Smoothing with Isotropic Noise
• Theorem 2: Asymmetric Randomized Smoothing via Universal Transformation
• proof
• Corollary 3: Asymmetric Randomized Smoothing with Independent Anisotropic Noise
• proof
• Corollary 4
• proof
• Definition 5: d-dimensional Generalized Super-ellipsoid of $\delta$
• Theorem 6: Lebesgue Measure of the Robust Perturbation Set $S(d,p)$
• proof