PiDAn: A Coherence Optimization Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks

TL;DR

A method based on subspace projective clustering (SPC), which learns a subspace as well as a projection-based weight vector by solving a projection maximization program, and the optimized weight vector can be utilized in a clustering framework to infer the group of data.

Abstract

Backdoor attacks impose a new threat in Deep Neural Networks (DNNs), where a backdoor is inserted into the neural network by poisoning the training dataset, misclassifying inputs that contain the adversary trigger. The major challenge for defending against these attacks is that only the attacker knows the secret trigger and the target class. The problem is further exacerbated by the recent introduction of "Hidden Triggers", where the triggers are carefully fused into the input, bypassing detection by human inspection and causing backdoor identification through anomaly detection to fail. To defend against such imperceptible attacks, in this work we systematically analyze how representations, i.e., the set of neuron activations for a given DNN when using the training data as inputs, are affected by backdoor attacks. We propose PiDAn, an algorithm based on coherence optimization purifying the poisoned data. Our analysis shows that representations of poisoned data and authentic data in the target class are still embedded in different linear subspaces, which implies that they show different coherence with some latent spaces. Based on this observation, the proposed PiDAn algorithm learns a sample-wise weight vector to maximize the projected coherence of weighted samples, where we demonstrate that the learned weight vector has a natural "grouping effect" and is distinguishable between authentic data and poisoned data. This enables the systematic detection and mitigation of backdoor attacks. Based on our theoretical analysis and experimental results, we demonstrate the effectiveness of PiDAn in defending against backdoor attacks that use different settings of poisoned samples on GTSRB and ILSVRC2012 datasets. Our PiDAn algorithm can detect more than 90% infected classes and identify 95% poisoned samples.

Figures (12)

• Figure 1: Illustration of not well shaped features/components: (a) distributions of first component and second component and (b) classification results by SVM.
• Figure 2: Plot of correlations for Spectral signatures tran2018spectral of clean data and poisoned data from (a) ILSVARC2012 (b) GTSRB. We calculate the correlation of spectral signatures with the top eigenvector of covariance matrix of mixture representations.
• Figure 3: An intuitive example to illustrate the insight of our algorithm. The blue cycle denotes authentic representation and the red cycle denotes poisoned representations. $\mathcal{S}_1$ and $\mathcal{S}_2$ are 2-dimensional subspaces in a $\mathbb{R}^3$ space and the orthogonal subspace of $\mathcal{S}_1$ is a vector line. We can observe each point in $\mathcal{S}_1$ is orthogonal to $\mathcal{S}_1^\perp$ and thus makes no contribution at all to increase the coherence.
• Figure 4: The square triggers used in the experiments.
• Figure 5: The optimized coefficients for (a) the infected class and (b) the clean class. The distribution of coefficients for the infected class follows a two Gaussian mixture, while distribution of coefficients for the clean class follows single Gaussian distribution.