Improving behavior based authentication against adversarial attack using XAI

TL;DR

This study explores a more practical scenario in behavior-based authentication, where adversarial samples are collected from the attacker and proposes an eXplainable AI (XAI) based defense strategy against adversarial attacks in such scenarios.

Abstract

In recent years, machine learning models, especially deep neural networks, have been widely used for classification tasks in the security domain. However, these models have been shown to be vulnerable to adversarial manipulation: small changes learned by an adversarial attack model, when applied to the input, can cause significant changes in the output. Most research on adversarial attacks and corresponding defense methods focuses only on scenarios where adversarial samples are directly generated by the attack model. In this study, we explore a more practical scenario in behavior-based authentication, where adversarial samples are collected from the attacker. The generated adversarial samples from the model are replicated by attackers with a certain level of discrepancy. We propose an eXplainable AI (XAI) based defense strategy against adversarial attacks in such scenarios. A feature selector, trained with our method, can be used as a filter in front of the original authenticator. It filters out features that are more vulnerable to adversarial attacks or irrelevant to authentication, while retaining features that are more robust. Through comprehensive experiments, we demonstrate that our XAI based defense strategy is effective against adversarial attacks and outperforms other defense strategies, such as adversarial training and defensive distillation.

Figures (9)

• Figure 1: Illustration of XAI augmented behavioral biometric authenticator. 4 mouse cursor movements (marked in blue) are selected as key part which is more important and robust than the rest 6 movements (marked in red) for user authentication. ① The feature selector takes an input x and returns a k-hot vector M which indicates whether each cognitive chunk, e.g. movement, will be selected as explanation for classification. ②$T=M\cdot X$ serves as a information bottleneck to get rid of the redundant and unreliable features. ③ The user authenticator takes $T=M\cdot X$ for classification. In this research, feature selection means selecting part of the whole sequence rather than selecting a subset of all extracted dynamic features like velocity, acceleration, angular velocity and so on.
• Figure 2: A mouse behavior based user authentication system.
• Figure 3: In our experiment, each input sample includes 10 consecutive movements that follows certain pattern. 10 adversarial attack models are trained to generate 10 movements correspondingly to maximize prediction loss. 1 out of 10 (feature selector not accessible) or 1 out of 2/3/4/5 (feature selector accessible so attackers know which 2/3/4/5 movements are selected) models that achieves the lowest true positive rate is selected as the final attacking model.
• Figure 4: Our improved feature selector training diagram.
• Figure 5: After we apply random Gaussian noise (purple bars) to the generated feature dimension of the adversary sample (purple dots), more differentiable and consistent feature dimension (feature X) becomes more robust to the adversarial attack (part of the space between bars is out of classifier boundaries (green lines) of the valid user).