Support is All You Need for Certified VAE Training

TL;DR

CIVET introduces a principled framework for certifiably robust training of Variational Autoencoders by reducing the challenge of bounding the worst-case loss over a distributional latent space to bounding a deterministic decoder over a latent-space support set. The method selects a latent subset from encoder outputs that captures most of the mass with respect to a target probability, and computes a differentiable upper bound on the decoder’s worst-case loss over that subset, enabling gradient-based optimization without imposing Lipschitz or fixed-variance constraints. By combining bounds over multiple such supports with a probabilistic weighting scheme, CIVET delivers improved certified robustness across wireless and vision tasks while maintaining competitive standard accuracy, outperforming Lipschitz-constrained VAEs and several baselines. The approach opens a path toward broader certified robustness for stochastic neural networks and complex generative models, with practical implications for safe deployment in safety-critical systems.

Abstract

Variational Autoencoders (VAEs) have become increasingly popular and deployed in safety-critical applications. In such applications, we want to give certified probabilistic guarantees on performance under adversarial attacks. We propose a novel method, CIVET, for certified training of VAEs. CIVET depends on the key insight that we can bound worst-case VAE error by bounding the error on carefully chosen support sets at the latent layer. We show this point mathematically and present a novel training algorithm utilizing this insight. We show in an extensive evaluation across different datasets (in both the wireless and vision application areas), architectures, and perturbation magnitudes that our method outperforms SOTA methods achieving good standard performance with strong robustness guarantees.

Figures (3)

• Figure 1: (CIVET Overview) The blue dashed box () shows a standard pass over a VAE, where the encoder ($N^{e}$) generates a parameterization of a distribution which is sampled in the latent layer and passed to the deterministic decoder ($N^{d}$). The green dotted box () shows CIVET training over the same VAE. Here an input region is passed through $N^{e}$ using a deterministic DNN bounding algorithm like IBP which gives a range of distribution parameterizations. CIVET then computes a support set with a given probability threshold $(1 - \delta)$ which can then be passed through $N^{d}$ using a deterministic DNN bounding algorithm to obtain an overapproximation of the loss.
• Figure 2: Support Set Visualization. Given a set of distributions $\{\mathcal{N}(\mu, \sigma)|\mu\in[\mu_{lb}, \mu_{ub}], \sigma\in[\sigma_{lb}, \sigma_{ub}]\}$ we define a symmetric support set $S_\delta = [\mu_{lb} - \zeta, \mu_{ub} + \zeta]$
• Figure 3: Standard and Certified SNR while varying $\mathcal{D}$. We consider sets with $\delta_n = 0.05$ and $\delta_i = \delta_{i+1} + \eta$ (let $n = |\mathcal{D}|$). On the left, we vary the size of $\mathcal{D}$ between 1 and 5 while fixing $\eta = 0.15$. On the right, we fix $|\mathcal{D}| = 3$ and vary $\eta$ between 0.05 and 0.25.

Theorems & Definitions (15)

• Definition 1: Support Sets
• Theorem 1
• Lemma 1
• Theorem 2
• Definition 2
• Theorem 2
• proof
• Lemma 2
• proof
• Lemma 3