Towards Robust Domain Generation Algorithm Classification

TL;DR

The paper addresses the robustness of deep learning–based DGA classifiers in adversarial settings and introduces 32 white-box attacks, novel discretization from embedding space to valid domains, and a joint adversarial training scheme. It reports substantial robustness gains from adversarial training, particularly when combining embedding-space and discretized-domain attacks, with a real-world evaluation showing time-robust performance and improved detection of unknown DGAs. A comprehensive leave-one-group-out evaluation demonstrates generalization to unseen attacks, while the study exposes training biases that AT can mitigate. The work provides a publicly available library of attacks and defenses to facilitate practical hardening of DGA classifiers and advocates using ensemble detection to achieve reliable security outcomes.

Abstract

In this work, we conduct a comprehensive study on the robustness of domain generation algorithm (DGA) classifiers. We implement 32 white-box attacks, 19 of which are very effective and induce a false-negative rate (FNR) of $\approx$ 100\% on unhardened classifiers. To defend the classifiers, we evaluate different hardening approaches and propose a novel training scheme that leverages adversarial latent space vectors and discretized adversarial domains to significantly improve robustness. In our study, we highlight a pitfall to avoid when hardening classifiers and uncover training biases that can be easily exploited by attackers to bypass detection, but which can be mitigated by adversarial training (AT). In our study, we do not observe any trade-off between robustness and performance, on the contrary, hardening improves a classifier's detection performance for known and unknown DGAs. We implement all attacks and defenses discussed in this paper as a standalone library, which we make publicly available to facilitate hardening of DGA classifiers: https://gitlab.com/rwth-itsec/robust-dga-detection

Figures (6)

• Figure 1: Minibatch layout of embedding-space AT.
• Figure 2: Minibatch layout of discrete domain AT.
• Figure 3: FNRs of the classifiers trained using different adversarial training schemes against the attacks we developed. The FNRs against the held-out attacks are framed in blue. None of the classifiers were trained on the black-box DGA samples.
• Figure 4: Averaged ROC curves across five folds of the real-world study. The mean ROC curves are additionally averaged over 46 evaluation runs.
• Figure 5: Success rates of the chosen attacks as a function of the attack strengths and discretization schemes.