Localized Randomized Smoothing for Collective Robustness Certification

TL;DR

The paper addresses collective robustness for multi-output models by introducing localized randomized smoothing, where each output is smoothed with anisotropic noise tailored to its input locality. A MILP-based collective certificate aggregates per-output base certificates, enabling provable bounds on how many predictions can be compromised by a single adversarial input under a budget constraint. Key contributions include a formal base-certificate interface, an efficient LP formulation, efficiency-enhancing strategies (sharing noise, partitioning, and binning), and variance-constrained certification for discrete data. Empirical results on semantic segmentation and graph node classification show that localized smoothing can achieve stronger average certifiable radii and competitive accuracy compared to isotropic baselines, particularly for softly local models. The work offers a general, scalable certificate for softly local multi-output tasks and highlights the role of locality in achieving cooperative robustness with practical implications for safer deployment of complex models.

Abstract

Models for image segmentation, node classification and many other tasks map a single input to multiple labels. By perturbing this single shared input (e.g. the image) an adversary can manipulate several predictions (e.g. misclassify several pixels). Collective robustness certification is the task of provably bounding the number of robust predictions under this threat model. The only dedicated method that goes beyond certifying each output independently is limited to strictly local models, where each prediction is associated with a small receptive field. We propose a more general collective robustness certificate for all types of models. We further show that this approach is beneficial for the larger class of softly local models, where each output is dependent on the entire input but assigns different levels of importance to different input regions (e.g. based on their proximity in the image). The certificate is based on our novel localized randomized smoothing approach, where the random perturbation strength for different input regions is proportional to their importance for the outputs. Localized smoothing Pareto-dominates existing certificates on both image segmentation and node classification tasks, simultaneously offering higher accuracy and stronger certificates.

Figures (14)

• Figure 1: Localized randomized smoothing applied to semantic segmentation. We assume that the most relevant information for labeling a pixel is contained in other nearby pixels. We partition the input image into multiple grid cells. For each grid cell, we sample noisy images from a different anisotropic distribution that applies more noise to far-away, less relevant cells. Segmenting all noisy images, cropping the result and computing the majority vote yields a local segmentation mask. These per-cell segmentation masks can then be combined into a complete segmentation mask.
• Figure 2: Comparison of isotropic smoothing with $\sigma_\mathrm{iso} \in \{0.01,\dots,0.5\}$ to our LP-based certificate with $\left(\sigma_\mathrm{min},\sigma_\mathrm{max}\right) = (\sigma_\mathrm{iso}, \infty)$, using a modified, strictly local U-Net on Pascal-VOC. Localized smoothing offers the same mIOU as SegCertify* and stronger robustness certificates.
• Figure 3: Certified accuracy of U-Net on Pascal-VOC. We compare SegCertify* ($\sigma_\mathrm{iso}=0.2$) to localized smoothing ($\left(\sigma_\mathrm{min},\sigma_\mathrm{max}\right) = (0.15, 1.0)$). Combining the base certificates (dashed blue line) via our collective LP (solid blue line) outperforms the baseline.
• Figure 4: Comparison of isotropic smoothing to our LP-based certificate with a $3 \times 5$ grid and U-Net on Pascal-VOC. U-Net is sufficiently local to benefit from localized smoothing (\ref{['fig:many_samples']}), but not enough to offset the increased sample complexity (\ref{['fig:few_samples']}) for the probabilistic base certificates.
• Figure 5: Comparison of our LP-based collective certificate to Bojchevski2020, using APPNP on Citeseer. We consider both adversarial deletions (\ref{['fig:citeseer_pm_appnp_deletion']}) and additions (\ref{['fig:citeseer_pm_appnp_addition']}) of attribute bits. Locally smoothed models offer a better accuracy-robustness tradeoff , especially for deletions. Transparent points signal that they are Pareto-dominated by points from the same method.

Theorems & Definitions (21)

• Definition 3.1: Base certificates
• Definition 4.1: Base certificate interface
• Theorem 4.2
• Theorem 5.1: Variance-constrained certification
• Theorem 4.2
• proof
• Proposition F.1
• proof
• Proposition F.2
• proof