Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Certified Robustness Under Bounded Levenshtein Distance

Elias Abad Rocamora, Grigorios G. Chrysos, Volkan Cevher

TL;DR

This work tackles robust textual classification under adversarial perturbations constrained by the Levenshtein distance. It introduces LipsLev, a Lipschitz-constant–based framework that uses ERP distance to bound the margin $g_{y,\hat{y}}$ of convolutional NLP models, enabling deterministic, single-pass certificates for perturbations of any length. By deriving a layer-wise Lipschitz bound and training 1-Lipschitz classifiers, LipsLev achieves non-trivial verified accuracy for $d_{\text{Lev}}\le 2$ (e.g., 38.8% at distance 1 and 13.93% at distance 2 on AG-News) and is orders of magnitude faster than brute-force or IBP baselines. This approach establishes a practical foundation for scalable, deterministic verification in NLP and opens avenues for extending Lipschitz verification to more complex architectures and tokenization schemes.

Abstract

Text classifiers suffer from small perturbations, that if chosen adversarially, can dramatically change the output of the model. Verification methods can provide robustness certificates against such adversarial perturbations, by computing a sound lower bound on the robust accuracy. Nevertheless, existing verification methods incur in prohibitive costs and cannot practically handle Levenshtein distance constraints. We propose the first method for computing the Lipschitz constant of convolutional classifiers with respect to the Levenshtein distance. We use these Lipschitz constant estimates for training 1-Lipschitz classifiers. This enables computing the certified radius of a classifier in a single forward pass. Our method, LipsLev, is able to obtain $38.80$% and $13.93$% verified accuracy at distance $1$ and $2$ respectively in the AG-News dataset, while being $4$ orders of magnitude faster than existing approaches. We believe our work can open the door to more efficient verification in the text domain.

Certified Robustness Under Bounded Levenshtein Distance

TL;DR

This work tackles robust textual classification under adversarial perturbations constrained by the Levenshtein distance. It introduces LipsLev, a Lipschitz-constant–based framework that uses ERP distance to bound the margin of convolutional NLP models, enabling deterministic, single-pass certificates for perturbations of any length. By deriving a layer-wise Lipschitz bound and training 1-Lipschitz classifiers, LipsLev achieves non-trivial verified accuracy for (e.g., 38.8% at distance 1 and 13.93% at distance 2 on AG-News) and is orders of magnitude faster than brute-force or IBP baselines. This approach establishes a practical foundation for scalable, deterministic verification in NLP and opens avenues for extending Lipschitz verification to more complex architectures and tokenization schemes.

Abstract

Text classifiers suffer from small perturbations, that if chosen adversarially, can dramatically change the output of the model. Verification methods can provide robustness certificates against such adversarial perturbations, by computing a sound lower bound on the robust accuracy. Nevertheless, existing verification methods incur in prohibitive costs and cannot practically handle Levenshtein distance constraints. We propose the first method for computing the Lipschitz constant of convolutional classifiers with respect to the Levenshtein distance. We use these Lipschitz constant estimates for training 1-Lipschitz classifiers. This enables computing the certified radius of a classifier in a single forward pass. Our method, LipsLev, is able to obtain % and % verified accuracy at distance and respectively in the AG-News dataset, while being orders of magnitude faster than existing approaches. We believe our work can open the door to more efficient verification in the text domain.
Paper Structure (24 sections, 10 theorems, 45 equations, 5 figures, 7 tables)

This paper contains 24 sections, 10 theorems, 45 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Interval Bound Propagation (IBP)
  5. Method
  6. Lipschitz constant based verification
  7. Lipschitz constant estimation for convolutional classifiers
  8. Training 1-Lipschitz classifiers
  9. Experiments
  10. Experimental setup
  11. Comparison with IBP and Brute Force approaches
  12. Regularizing the Lipschitz constant
  13. The influence of sentence length in verification
  14. Conclusion
  15. Additional experimental validation
  16. ...and 9 more sections

Key Result

Theorem 4.3

Let $\bm{f}$ be defined as in eq:conv_model. Let $g_{y,\hat{y}}(\bm{S}) = f(\bm{S})_{y} - f(\bm{S})_{\hat{y}}$ be the margin function for classes $y$ and $\hat{y}$. Let $p\geq 1$. Let $\bm{P}$ and $\bm{Q}$ be sequences of one-hot vectors, we have that for any $y$ and $\hat{y}$: where $M(\bm{\mathcal{K}})=\sum_{i=1}^{q}\left|\left|\bm{K}_i\right|\right|_{p}$, $M(\bm{E}) = \max\{\underset{i\in [|\G

Figures (5)

  • Figure 1: Sentence length distribution for verified and not verified sentences: We report the verified accuracy v.s. sentence length with LipsLev(Left), and the average length of verified and not verified sentences (Right) at $k=1$ in the models trained with $p=2$. Shorter sentences are harder to verify in both SST-2 and AG-News with both LipsLev and the brute-force approach.
  • Figure S2: Sentence length distribution for verified and not verified sentences: We report the histogram of the lengths for verified and not verified sentences at $k=1$ with LipsLev in the models trained with $p=2$. Shorter sentences are harder to verify in both SST-2 and AG-News with both LipsLev and the brute force approach.
  • Figure S3: Learning rate selection for the SST-2 and AG-News datasets: We report the clean and verified accuracy in a validation set of 1,000 sentences extracted from the training split of each dataset and set aside during training. We set the learning rate equal to $100$ in the rest of our experiments as it provides a good trade-off between clean and verified accuracy for all norms and datasets.
  • Figure S4: Training deeper models in AG-News: We report the clean and verified accuracies with LipsLev at $k=1$ for $p\in \{1,2,\infty\}$. Clean and verified accuracies decrease with the number of layers. With $p=2$ the performance is less degraded with the number of layers.
  • Figure S5: Training deeper models in SST-2: We report the clean and verified accuracies with LipsLev at $k=1$ for $p\in \{1,2,\infty\}$. Clean and verified accuracies decrease with the number of layers. With $p=2$ the performance is less degraded with the number of layers.

Theorems & Definitions (29)

  • Definition 4.1: ERP distance chen2004ERP
  • Definition 4.2: 1D Convolutional layer with zero padding
  • Theorem 4.3: Lipschitz constant of margins of convolutional models
  • proof
  • Corollary 4.4: Certified radius of convolutional models
  • Remark 4.5: Local Lipschitz constant of the embedding layer
  • Corollary 4.6: Local Lipschitz constant of modified classifiers
  • Definition S1: Clean accuracy
  • Definition S2: Adversarial accuracy
  • Definition S3: Verified accuracy
  • ...and 19 more