Activation Gradient based Poisoned Sample Detection Against Backdoor Attacks

TL;DR

This work introduces gradient circular distribution (GCD), a novel activation-gradient perspective, to distinguish poisoned and clean samples in backdoor defense. Building on GCD, the authors propose Activation Gradient based Poisoned Detection (AGPD), a three-stage PSD pipeline that identifies target classes via dispersion metrics and then filters poisoned samples using a sample-level closeness score, with stopping criteria based on Jensen–Shannon divergence. Empirical results across CIFAR-10, Tiny ImageNet, and additional datasets show AGPD achieving high true-positive rates and near-zero false positives across all-to-one, all-to-all, and multi-target backdoor attacks, outperforming activation-, input-, and loss-based detectors. The method demonstrates robustness to varying poisoning ratios and model architectures, offering a scalable, discriminative approach for data-poisoning defense, with plans to improve efficiency by restricting the layer search range.

Abstract

This work studies the task of poisoned sample detection for defending against data poisoning based backdoor attacks. Its core challenge is finding a generalizable and discriminative metric to distinguish between clean and various types of poisoned samples (e.g., various triggers, various poisoning ratios). Inspired by a common phenomenon in backdoor attacks that the backdoored model tend to map significantly different poisoned and clean samples within the target class to similar activation areas, we introduce a novel perspective of the circular distribution of the gradients w.r.t. sample activation, dubbed gradient circular distribution (GCD). And, we find two interesting observations based on GCD. One is that the GCD of samples in the target class is much more dispersed than that in the clean class. The other is that in the GCD of target class, poisoned and clean samples are clearly separated. Inspired by above two observations, we develop an innovative three-stage poisoned sample detection approach, called Activation Gradient based Poisoned sample Detection (AGPD). First, we calculate GCDs of all classes from the model trained on the untrustworthy dataset. Then, we identify the target class(es) based on the difference on GCD dispersion between target and clean classes. Last, we filter out poisoned samples within the identified target class(es) based on the clear separation between poisoned and clean samples. Extensive experiments under various settings of backdoor attacks demonstrate the superior detection performance of the proposed method to existing poisoned detection approaches according to sample activation-based metrics.

Figures (12)

• Figure 1: Gradient circular distributions (GCDs) across four classes of CIFAR-10, on the clean model (left), Blended attacked model (middle), and SSBA attacked model (right), respectively. The value along with each arc indicates the CVBT value. The GCD of the target class (covering both black and blue arcs). Note that we moved three clean classes' arcs to different quadrants to avoid visual overlap.
• Figure 2: Illustrations of gradient circular distribution (GCD) and two metrics, and the pipeline of the proposed APGD method which consists of three stages: 1) calculating activation gradient distribution, 2) identifying target class(es), and 3) filtering out poisoned samples within the identified target class(es).
• Figure 3: Detection performance of AGPD and the compared detectors with poisoing ratios ranging from $0.5\%$ to $10\%$.
• Figure 4: Statistical analysis of $\rho$ and $z$ across classes and convolutional layers using the CIFAR-10 and Preact-ResNet18. (a)$\rho$ values for all classes in both all-to-one and all-to-all attacks. (b)$z$ for all-to-one attacks. (c) Mean and standard deviation of the maximum $z$ across all layers in multiple backdoored models.
• Figure 5: left: Accuracy of AGPD on identifying target class(es) compared with three compared methods. middle: Detection performance of AGPD with varying numbers of clean samples. right: Means and standard deviations of TPR and FPR at different threshold $\tau_s$.

Theorems & Definitions (2)

• Definition 1: Activiation Gradient
• Definition 2: Gradient Circular Distribution (GCD)