$DA^3$: A Distribution-Aware Adversarial Attack against Language Models

TL;DR

DA^3 uncovers distribution shifts in adversarial NLP examples, notably in MSP and MD, which undermine detection-based stealth. It introduces Data Alignment Loss to train a LoRA-based PLM so generated adversaries mimic originals in MSP and MD, while an inference-time masking strategy yields effective attacks. The NASR metric integrates detectability with attack success to evaluate realism and stealth. Empirical results across four datasets show strong white-box performance and superior transferability to LLMs like LLaMA2-7b, with human evaluators confirming grammar quality and semantic preservation. Overall, the approach advances threat modeling and defense research by aligning adversarial generation with distributional properties of real data.

Abstract

Language models can be manipulated by adversarial attacks, which introduce subtle perturbations to input data. While recent attack methods can achieve a relatively high attack success rate (ASR), we've observed that the generated adversarial examples have a different data distribution compared with the original examples. Specifically, these adversarial examples exhibit reduced confidence levels and greater divergence from the training data distribution. Consequently, they are easy to detect using straightforward detection methods, diminishing the efficacy of such attacks. To address this issue, we propose a Distribution-Aware Adversarial Attack ($DA^3$) method. $DA^3$ considers the distribution shifts of adversarial examples to improve attacks' effectiveness under detection methods. We further design a novel evaluation metric, the Non-detectable Attack Success Rate (NASR), which integrates both ASR and detectability for the attack task. We conduct experiments on four widely used datasets to validate the attack effectiveness and transferability of adversarial examples generated by $DA^3$ against both the white-box BERT-base and RoBERTa-base models and the black-box LLaMA2-7b model.

Figures (14)

• Figure 1: Toy examples of two adversarial sentences in a sentiment analysis task. Although both sentences successfully attack the victim model, the top one is flagged by the detector, while the bottom one is not detected. In our task, we aim to generate adversarial examples that are hard to detect.
• Figure 2: Visualization of the distribution shift between original data and adversarial data generated by BERT-Attack when attacking BERT-base regarding MSP.
• Figure 3: Visualization of the distribution shift between original data and adversarial data generated by BERT-Attack when attacking BERT-base regarding MD.
• Figure 4: The model architecture of DA$^3$ comprises two phases: fine-tuning and inference. During fine-tuning, a LoRA-based PLM is fine-tuned to develop the ability to generate adversarial examples resembling original examples in terms of MSP and MD. During inference, the LoRA-based PLM is used to generate adversarial examples.
• Figure 5: The change of $\mathcal{L}_{MSP}$, $\mathcal{L}_{MD}$, and $\mathcal{L}_{DAL}$ throughout the fine-tuning phase of DA$^3$ with BERT-base as backbone on SST-2. The x-axis represents fine-tuning steps; the y-axis represents the change of loss compared to the initial loss.