Bridging the Theoretical Gap in Randomized Smoothing

TL;DR

This work addresses the gap between empirical robustness and theoretical certificates in randomized smoothing by introducing a Lipschitz-aware certification framework and an efficient class-partitioning method for exact coverage intervals. It derives new certified radii that incorporate local Lipschitz constants, yielding $R_{ ext{multiLip}}$ and $R_{ ext{monoLip}}$ that tighten robustness guarantees relative to standard RS radii. The core contribution is the Class Partitioning Method (CPM), which reduces conservativeness in multi-class confidence bounds by adaptively grouping classes and applying Bonferroni corrections at the bucket level. Experimental results on ImageNet and CIFAR-10 show CPM improves interval tightness and Lipschitz-based certificates align more closely with empirical robustness, highlighting the importance of local smoothness in bridging theory and practice.

Abstract

Randomized smoothing has become a leading approach for certifying adversarial robustness in machine learning models. However, a persistent gap remains between theoretical certified robustness and empirical robustness accuracy. This paper introduces a new framework that bridges this gap by leveraging Lipschitz continuity for certification and proposing a novel, less conservative method for computing confidence intervals in randomized smoothing. Our approach tightens the bounds of certified robustness, offering a more accurate reflection of model robustness in practice. Through rigorous experimentation we show that our method improves the robust accuracy, compressing the gap between empirical findings and previous theoretical results. We argue that investigating local Lipschitz constants and designing ad-hoc confidence intervals can further enhance the performance of randomized smoothing. These results pave the way for a deeper understanding of the relationship between Lipschitz continuity and certified robustness.

Figures (10)

• Figure 1: Illustration of the initial phase of the class partitioning method (CPM). The number of counts in class $I$ is noted $\mid I \mid$.
• Figure 2: Comparison of radii as a function of ${\bm{p}}_1$, for $L(F)=4$, $\sigma=0.12$ and ${\bm{p}}_2 = 0.1$.
• Figure 3: Comparison of various confidence interval methods for certified accuracy estimation with smoothing standard deviation $\sigma=0.5$ on the ImageNet dataset, using ResNet-50 trained with noise injection ($\sigma=0.5$). The plot contrasts our CPM method with Bonferroni and Pearson-Clopper intervals across different perturbation levels $\epsilon$.
• Figure 4: LiResNet trained with noise injection ($\sigma=0.5$), on CIFAR-10 dataset. Certified accuracy $R_{\mathrm{multi}}$ vs Lipschitz estimated certified accuracy $R_{\mathrm{multiLip}}$ vs empirical robust accuracy using projected gradient ascent. Randomized smoothing noise is taken as ($\sigma=0.12$).
• Figure 5: Comparison of various confidence interval methods for certified accuracy estimation with smoothing standard deviation $\sigma=0.12$ on the CIFAR-10 dataset.

Theorems & Definitions (10)

• Definition 1
• Theorem 1: hochberg1987multiple
• Theorem 2: Randomized Smoothing with Local Lipschitz Continuity
• Lemma 1
• Theorem 3: Pontryagin's Maximum Principle
• Lemma 2
• proof
• proof
• Remark 1
• proof