AdaptDel: Adaptable Deletion Rate Randomized Smoothing for Certified Robustness

TL;DR

This work targets certified robustness for sequence classification under edit-distance perturbations by moving beyond fixed-rate deletion smoothing. It introduces AdaptDel, an input-length dependent deletion mechanism, and AdaptDel+, which adds calibrated binning and optimization to maximize certified radius while maintaining clean accuracy. The authors extend randomized smoothing theory to variable-rate deletions, deriving tractable bounds via longest common subsequences and knapsack-style optimizations, and they demonstrate substantial gains across four NLP tasks, especially for longer inputs. The results show large improvements in robustness (mean CR and median CC) with modest trade-offs in clean accuracy, highlighting the practical impact of input-adaptive smoothing for real-world, variable-length sequences.

Abstract

We consider the problem of certified robustness for sequence classification against edit distance perturbations. Naturally occurring inputs of varying lengths (e.g., sentences in natural language processing tasks) present a challenge to current methods that employ fixed-rate deletion mechanisms and lead to suboptimal performance. To this end, we introduce AdaptDel methods with adaptable deletion rates that dynamically adjust based on input properties. We extend the theoretical framework of randomized smoothing to variable-rate deletion, ensuring sound certification with respect to edit distance. We achieve strong empirical results in natural language tasks, observing up to 30 orders of magnitude improvement to median cardinality of the certified region, over state-of-the-art certifications.

Figures (8)

• Figure 1: Certified accuracy plotted against a lower bound on the log-cardinality of the certified region. Each point $(c, a)$ on a curve indicates that a fraction $a$ of the test inputs were correctly classified with a certified log-cardinality of at least $c$. While AdaptDel consistently outperforms the baselines, AdaptDel+ achieves the highest certified accuracy for larger certified regions.
• Figure 2: Certified accuracy as a function of the log-cardinality of the certified region, grouped by quartile of input size. The subfigure on the right displays the quantile by input size, with the dashed lines indicating the quartiles corresponding to the certified accuracy plots on the left. Each set of axes (top to bottom) corresponds to a split of the test set on the length-based quartiles (smallest to largest). For example, the second plot from top to bottom shows the certified accuracies of examples within Q1 to Q2. The results demonstrate that the methods scale effectively across varying input sizes, with higher certified accuracy achieved for larger input sizes and higher log-cardinality regions.
• Figure 3: Plots of the deletion rate $\psi(\bm{x})$ for AdaptDel and AdaptDel+ and the retention length $k_{g(\bm{x})}$ for AdaptDel+. The top left plot shows the deletion rates $\psi(\bm{x})$ for AdaptDel as a function of input length. The top right plot shows the deletion rates $\psi(\bm{x})$ for AdaptDel+ as a function of input length. The bottom left plot shows the retention rates $k_{g(\bm{x})}$ for AdaptDel+ as a function of input length. The plots demonstrate that AdaptDel and AdaptDel+ adaptively adjust their deletion rates based on the input length.
• Figure 4: Certified accuracy as a function of the log-cardinality of the certified region, grouped by quartile of input size. The subfigure on the right displays the quantile by input size, with the dashed lines indicating the quartiles corresponding to the certified accuracy plots on the left. Each set of axes (top to bottom) corresponds to a split of the test set on the length-based quartiles (smallest to largest). For example, the second plot from top to bottom shows the certified accuracies of examples with in Q1 to Q2. The results demonstrate that the methods scale effectively across varying input sizes, with higher certified accuracy achieved for larger input sizes and higher log-cardinality regions.
• Figure 5: Certified accuracy for all methods as a function of the certified radius. While AdaptDel consistently outperforms RanMASK and CERT-ED across all radii, AdaptDel+ achieves the highest certified accuracy for larger certified regions. Note that, CERT-ED, AdaptDel, and AdaptDel+ certifies Leveshtein distance perturbations, while RanMASK only certifies Hamming distance perturbations. Thus, the actual robustness of RanMASK at the same radii is lower than that of CERT-ED, AdaptDel and AdaptDel+.

Theorems & Definitions (13)

• remark 1: On Smoothing vs. Certified Edits
• lemma 1: note=huang2023rsdel, store=lem:equiv-edit-del, label=lem:equiv-edit-del
• lemma 2: store=lem:var-del-cert-pairwise-lb, label=lem:var-del-cert-pairwise-lb
• proof : Proof sketch
• theorem 1: store=thm:certify-length-dep, label=thm:certify-length-dep
• lemma 3
• proof
• proof
• lemma 4
• proof