Robustness Certificates for Neural Networks against Adversarial Attacks

TL;DR

This work tackles the lack of formal guarantees for robustness against data poisoning in safety-critical ML by modeling gradient-based training as a discrete-time dynamical system and reframing poisoning robustness as a safety verification problem using barrier certificates. To overcome intractability in high dimensions, it proposes a neural network barrier certificate (NNBC) learned from poisoned trajectories and validated with a scenario convex program (SCP) that yields PAC-style guarantees on a certified robust radius. The framework is model- and attack-agnostic, providing unified certificates for both train-time and test-time perturbations without requiring prior knowledge of the attack or contamination level, and it demonstrates non-trivial certifiable budgets on MNIST, SVHN, and CIFAR-10. The approach advances formal safety guarantees for learning under adversarial data, with practical implications for reliable deployment in real-world systems, and suggests directions for extending to broader threat models and efficiency improvements.

Abstract

The increasing use of machine learning in safety-critical domains amplifies the risk of adversarial threats, especially data poisoning attacks that corrupt training data to degrade performance or induce unsafe behavior. Most existing defenses lack formal guarantees or rely on restrictive assumptions about the model class, attack type, extent of poisoning, or point-wise certification, limiting their practical reliability. This paper introduces a principled formal robustness certification framework that models gradient-based training as a discrete-time dynamical system (dt-DS) and formulates poisoning robustness as a formal safety verification problem. By adapting the concept of barrier certificates (BCs) from control theory, we introduce sufficient conditions to certify a robust radius ensuring that the terminal model remains safe under worst-case ${\ell}_p$-norm based poisoning. To make this practical, we parameterize BCs as neural networks trained on finite sets of poisoned trajectories. We further derive probably approximately correct (PAC) bounds by solving a scenario convex program (SCP), which yields a confidence lower bound on the certified robustness radius generalizing beyond the training set. Importantly, our framework also extends to certification against test-time attacks, making it the first unified framework to provide formal guarantees in both training and test-time attack settings. Experiments on MNIST, SVHN, and CIFAR-10 show that our approach certifies non-trivial perturbation budgets while being model-agnostic and requiring no prior knowledge of the attack or contamination level.

Figures (7)

• Figure 1: Overview of our proposed framework against train-time attacks. (Left) Data Generation: The model $h_\theta$ is trained on multiple poisoned datasets with varying perturbation levels to generate a set of parameter trajectories. The terminal parameters are labeled as safe or unsafe based on the test accuracy degradation. (Right) Certification: A Neural Network-based Barrier Certificate (NNBC) $\mathcal{B}$ is learned from these parameters. The validity of $\mathcal{B}$ is then rigorously verified via a PAC bound guarantee, providing a certified robust radius with a violation probability of at most $\epsilon$ and a confidence of at least $1\! -\! \beta$.
• Figure 2: Certified accuracy ($g_{\mathrm{p}}^*$) versus perturbation magnitude ($\delta$) on different settings and poisoning scenarios. Each figure reports the terminal test accuracy $g(\theta(t_\infty))$, the empirical robust radius $\delta_\mathrm{emp}$, and the certified robust radius $\delta^*_\mathrm{cert}$ obtained using our proposed framework. The confidence level is fixed at $1-\beta$, $\beta = 10^{-4}$, across all settings, with the corresponding violation probabilities being (a) $0.015$, (b) $0.013$, (c) $0.006$, and (d) $0.005$.
• Figure 3: Comparing the result of our framework and RAB on SVHN under test-time BDA with $\ell_\infty$ attack. Our results consistently yield higher certified robustness than RAB.
• Figure 4: Our proposed framework. The left panel illustrates the data generation process under both train-time poisoning attacks or test-time evasion attacks. For each perturbation level, the model $h_\theta$ is trained on perturbed datasets to produce two disjoint sets of parameter vectors: $\theta$ and $\hat{\theta}$. A safety criterion function is then applied to each parameter vector to label it as safe or unsafe. The set $\theta$ is used to train an NNBC $\mathcal{B}_\varphi$, while the set $\hat{\theta}$ is used to evaluate $\mathcal{B}_\varphi$ through a scenario-based PAC analysis. The certification process (right panel) outputs a certified NNBC $\mathcal{B}_{\phi}^*$, and its corresponding robustness radius $\delta^*_{\mathrm{cert}}$ or $\delta'^{*}_{\mathrm{cert}}$, and a probabilistic guarantee with violation probability at most $\epsilon$ and a confidence of at least $1\! -\! \beta$.
• Figure 5: Certified accuracy versus perturbation magnitude $\delta$ under different poisoning scenarios and datasets. Each subplot shows the test accuracy $g$, empirical robust radius $\delta_\mathrm{emp}$, and certified robust radius $\delta^*_\mathrm{cert}$ under the proposed framework. The confidence level is fixed at $99.99\%$. Violation probabilities are: $\epsilon =$ (a) $0.005$, (b) $0.011$, (c) $0.045$, (d) $0.003$, (e) $0.006$, (f) $0.006$, (g) $0.006$, (h) $0.004$, (i) $0.011$, (j) $0.015$, (k) $0.006$, (l) $0.045$, (m) $0.009$, (n) $0.003$, (o) $0.004$, (p) $0.009$, (q) $0.015$, (r) $0.015$, (s) $0.015$, (t) $0.022$.

Theorems & Definitions (14)

• Definition 2.1: ML Model
• Definition 2.2: Train-Time Poisoning attack
• Definition 2.3: Test-Time Evasion attack
• Definition 2.4: Model Degradation
• Remark 2.5
• Definition 2.7: Dynamical System
• Definition 2.8: Safety Specification
• Definition 2.9: Barrier Certificate
• Remark 2.10
• Remark 2.11