Instance-Level Trojan Attacks on Visual Question Answering via Adversarial Learning in Neuron Activation Space

TL;DR

The paper addresses the vulnerability of Visual Question Answering systems to Trojan attacks by introducing an instance-level multimodal Trojan that operates through a fixed perturbation layer. It leverages two targeted perturbation neurons and dual-modality adversarial learning to link their overactivations to malicious outputs, enabling transfer to fine-tuned models with very few Trojan samples. Extensive experiments on VQA-v2 show enhanced transferability, stealthiness, and sample efficiency, while conventional defenses like Differential Privacy and Norm Difference Estimation offer limited mitigation. The approach highlights practical risk in multimodal models and motivates the development of robust defenses for VQA and related architectures.

Abstract

Trojan attacks embed perturbations in input data leading to malicious behavior in neural network models. A combination of various Trojans in different modalities enables an adversary to mount a sophisticated attack on multimodal learning such as Visual Question Answering (VQA). However, multimodal Trojans in conventional methods are susceptible to parameter adjustment during processes such as fine-tuning. To this end, we propose an instance-level multimodal Trojan attack on VQA that efficiently adapts to fine-tuned models through a dual-modality adversarial learning method. This method compromises two specific neurons in a specific perturbation layer in the pretrained model to produce overly large neuron activations. Then, a malicious correlation between these overactive neurons and the malicious output of a fine-tuned model is established through adversarial learning. Extensive experiments are conducted using the VQA-v2 dataset, based on a wide range of metrics including sample efficiency, stealthiness, and robustness. The proposed attack demonstrates enhanced performance with diverse vision and text Trojans tailored for each sample. We demonstrate that the proposed attack can be efficiently adapted to different fine-tuned models, by injecting only a few shots of Trojan samples. Moreover, we investigate the attack performance under conventional defenses, where the defenses cannot effectively mitigate the attack.

Figures (8)

• Figure 1: A comparison between the dual-key backdoor dualkey and our method. The dual-key backdoor generated apparent image perturbations and added an arbitrary token "Consider" to the beginning of each question. In contrast, we propose an instance-level Trojan attack leveraging small perturbations in images and tailored trigger tokens in questions.
• Figure 2: The proposed Trojan attack utilizes the perturbation layer to mount adversarial learning within the activation space of two specific neurons. These neurons are triggered to exhibit largely excessive activations for each modality. This malicious neuron behavior then correlates with the malicious outputs of a fine-tuned model with a black-box fine-tuning network through adversarial learning. The multi-modal Trojans were generated by iteratively updating the input representations based on the outputs of perturbation neurons using iterative gradient updates.
• Figure 3: Neuron activations in the perturbation layer were visualized by reshaping the 1024-dimensional activation vectors to a dimension of $32 \times 32$. Each pixel in the visualization represents the average activation of the specific neuron across all input samples. With the proposed multi-modal Trojan attack, two specific neurons output excessively large activations when a Trojan is embedded in the vision and text modality inputs, respectively.
• Figure 4: Neuron activations in the perturbation layer with varying $\alpha$ and $E$.
• Figure 5: Distributions of vision and text modality input samples with and without the Trojans embedded.