RS-Reg: Probabilistic and Robust Certified Regression Through Randomized Smoothing

TL;DR

The paper extends randomized smoothing to regression by introducing a probabilistic robustness certificate for multivariate, continuous outputs. It develops an averaging-based RS (g(x) = E[f_θ(x + e)]) and provides theoretical guarantees: a base-model certificate under ell_2 perturbations (Theorem 1), asymptotic certificates for the smoothing estimator (Theorems 2 and 3), and a finite-sample, discounted certificate (Proposition 1). Empirical results on synthetic functions and a camera re-localization task validate the theory and illustrate practical trade-offs between robustness probability, smoothing level, and output bounds. The work offers a principled framework for certifiable regression under adversarial perturbations with guidance for bounded outputs and finite-sample use cases.

Abstract

Randomized smoothing has shown promising certified robustness against adversaries in classification tasks. Despite such success with only zeroth-order access to base models, randomized smoothing has not been extended to a general form of regression. By defining robustness in regression tasks flexibly through probabilities, we demonstrate how to establish upper bounds on input data point perturbation (using the $\ell_2$ norm) for a user-specified probability of observing valid outputs. Furthermore, we showcase the asymptotic property of a basic averaging function in scenarios where the regression model operates without any constraint. We then derive a certified upper bound of the input perturbations when dealing with a family of regression models where the outputs are bounded. Our simulations verify the validity of the theoretical results and reveal the advantages and limitations of simple smoothing functions, i.e., averaging, in regression tasks. The code is publicly available at \url{https://github.com/arekavandi/Certified_Robust_Regression}.

Figures (9)

• Figure 1: The general schematic of probabilistic certified robustness in regression where input can deviate from $\textbf{x}$ in any direction (bounded with respect to $\text{diss}_{x}$ used in $\mathbf{N}_{x}(\textbf{x},\epsilon_x)$) and the desired output should be within a range (with respect to $\text{diss}_{y}$) with probability $P$ where outputs are analyzed in $l$ groups (left). A particular case where the dissimilarity functions are $\ell_2$ norm and $l=t/2$ (right).
• Figure 2: The two-dimensional function used for the simulation (left). Theoretical certificates derived for $f(\textbf{x})$ (blue), $g(\textbf{x})$ for well-behaved base regression (red), and discounted certificates of $g(\textbf{x})$ (black and green) where the user set $P=80\%$ (right).
• Figure 3: Empirical probability of valid output in comparison with desired probability defined by the user (80%) for $f(\textbf{x})$ (top left), $g(\textbf{x})$ (top right), discounted $g(\textbf{x})$ (bottom).
• Figure 4: Certified median (left) and mean (right) error in DSAC$^{*}$ as a function of $r$.
• Figure 5: Examples of adversarial images contaminated with noise. As shown the changes in the certified range and considered noise are not visible to the human vision system, unless they become significant in magnitude.

Theorems & Definitions (4)

• proof
• proof : Proof of Theorem 2
• proof
• proof