Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks

TL;DR

The work develops a theoretical framework linking weight perturbations to output changes in deep networks, deriving exact minimal perturbations for single-layer updates and a margin-Lipschitz bound that extends to multi-layer perturbations and arbitrary architectures. It applies these results to structured perturbations from compression, showing that low-rank approximations can reliably activate latent backdoors while preserving full-precision accuracy, and provides certifiable thresholds for when such backdoors can emerge. The authors validate theory with experiments across vision models and language models, and demonstrate weight-modification conditioned backdoors, quantifying thresholds via Lipschitz estimates. The findings highlight a practical risk of efficiency-driven model compression enabling adversarial behavior and propose a principled direction for defenses based on parameter-space Lipschitz constants and perturbation analysis.

Abstract

The minimal norm weight perturbations of DNNs required to achieve a specified change in output are derived and the factors determining its size are discussed. These single-layer exact formulae are contrasted with more generic multi-layer Lipschitz constant based robustness guarantees; both are observed to be of the same order which indicates similar efficacy in their guarantees. These results are applied to precision-modification-activated backdoor attacks, establishing provable compression thresholds below which such attacks cannot succeed, and show empirically that low-rank compression can reliably activate latent backdoors while preserving full-precision accuracy. These expressions reveal how back-propagated margins govern layer-wise sensitivity and provide certifiable guarantees on the smallest parameter updates consistent with a desired output shift.

Figures (5)

• Figure 1: Decision boundaries before and after perturbing the final-layer weight by various amounts. The dotted lines show the unperturbed classifier assigning sample point $\mathbf{x}$ (black dot) to class 0; the solid colour is the boundaries after the weight perturbation.
• Figure 2: Perturbation norm vs. layer(s) perturbed and a lower bound on the perturbation size using the Margin-Lipschitz bound. Group perturbations (blue) unfreeze layers 1 through k; single-layer perturbations (orange) unfreeze only the k-th layer.
• Figure 3: Weight-modification-conditioned backdoor: a model is trained so that full-precision behaviour is preserved, while a compression map $g(\cdot)$ (e.g. pruning) activates a hidden backdoor.
• Figure 4: Perturbation norm vs. layer(s) perturbed and a lower bound on the perturbation size of a single layer of a network with ReLU activations using the Margin-Lipschitz bound. Group perturbations (blue) unfreeze layers 1 through k; single-layer perturbations (orange) unfreeze only the k-th layer.
• Figure 5: Perturbation norm vs. layer(s) perturbed and a lower bound on the perturbation size of a single layer of a network with tanh activations using the Margin-Lipschitz bound. Group perturbations (blue) unfreeze layers 1 through k; single-layer perturbations (orange) unfreeze only the k-th layer.

Theorems & Definitions (12)

• Definition 2.2: Output logit margin
• Definition 2.3: Layer-$N$ pre-image difference
• Theorem 3.1: Perturbation in the $N$th Layer for an $M$-Layer Network with a Locally Invertible Downstream Map
• Remark 3.2: Row--space condition
• Theorem 4.1: Robustness to Parameter Perturbations under the $\ell_p$ Norm
• Corollary 5.1: Final-layer low-rank projection induced misclassification.
• Remark 5.2: Consistency with Margin–Robustness bound
• Theorem A.1: Perturbation in the $N$th Layer for an $M$-Layer Network with a Locally Invertible Downstream Map Under Rank-$k$ Target Restriction
• proof
• Theorem A.2