RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalance

TL;DR

This work addresses the heightened vulnerability of deep neural networks to backdoor attacks in imbalanced datasets, showing that majority-class bias increases attack success and degrades existing defenses. It introduces Randomized Probability Perturbation (RPP), a certified poisoned-sample detector that operates on per-sample prediction stability under random perturbations and uses conformal prediction to calibrate decision thresholds in a black-box setting. Theoretical guarantees provide a detectable trigger-size bound and false-positive control, while extensive experiments on five benchmarks and ten attack types demonstrate strong detection and certification performance, especially under imbalance. The approach offers a practical, principled foundation for defending against backdoor threats in real-world, skewed data environments, with resilience to adaptive attacks and varying noise conditions.

Abstract

Deep neural networks are highly susceptible to backdoor attacks, yet most defense methods to date rely on balanced data, overlooking the pervasive class imbalance in real-world scenarios that can amplify backdoor threats. This paper presents the first in-depth investigation of how the dataset imbalance amplifies backdoor vulnerability, showing that (i) the imbalance induces a majority-class bias that increases susceptibility and (ii) conventional defenses degrade significantly as the imbalance grows. To address this, we propose Randomized Probability Perturbation (RPP), a certified poisoned-sample detection framework that operates in a black-box setting using only model output probabilities. For any inspected sample, RPP determines whether the input has been backdoor-manipulated, while offering provable within-domain detectability guarantees and a probabilistic upper bound on the false positive rate. Extensive experiments on five benchmarks (MNIST, SVHN, CIFAR-10, TinyImageNet and ImageNet10) covering 10 backdoor attacks and 12 baseline defenses show that RPP achieves significantly higher detection accuracy than state-of-the-art defenses, particularly under dataset imbalance. RPP establishes a theoretical and practical foundation for defending against backdoor attacks in real-world environments with imbalanced data.

Figures (12)

• Figure 1: AUC/ASR of Badnets on MNIST with balanced ($\rho=1$) and imbalanced ($\rho=2,10,100,200$) training sets; long-tailed ($r=45$, $p=0.4\%$) and step-imbalanced ($r=18$, $p=0.3\%$).
• Figure 2: Relative frequency distribution of $\tilde{\Delta}P$ on the SVHN dataset with $\mu = 0.9$ under imbalance ratios: (a) $\rho = 2$ and (b) $\rho = 100$.
• Figure 3: Overview of RPP method.
• Figure 4: Performance of RPP against BadNets attacks with perturbation magnitude $\|\delta\|_2 \geq 0.8$, measured by TPR and FPR across balanced ($\rho = 1$) and imbalanced ($\rho = 2, 10, 100, 200$) settings, with $n = 100$ and varying $\alpha$ on SVHN, CIFAR-10, TinyImageNet and ImageNet10.
• Figure 5: Performance of RPP against BadNets attacks with $\|\delta\|_2 \geq 0.8$, measured by TPR and FPR on SVHN, CIFAR-10, TinyImageNet and ImageNet10 under balanced ($\rho = 1$) and imbalanced ($\rho = 2, 10, 100, 200$) settings for varying $\sigma$, with $\alpha = 0.05$ and $n = 100$.

Theorems & Definitions (11)

• Definition 4.1: Samplewise Probability Vector (SPV)
• Definition 4.2: Randomized Probability Perturbation (RPP) and empirical RPP
• Theorem 5.1
• Corollary 5.2
• Theorem 5.3
• Theorem 5.4
• Theorem A.1: \ref{['them:main_1']}
• Lemma A.2: Lemma 4, cohen2019certified
• proof : Proof of \ref{['them:restated_main_1']}
• Lemma A.4