On the Adversarial Robustness of Multi-Modal Foundation Models

TL;DR

The paper analyzes the adversarial robustness of multi-modal vision-language foundation models, focusing on OpenFlamingo. It introduces a white-box APGD-based attack framework that perturbs visual inputs within small ℓ∞ radii to influence both captioning and VQA outputs, evaluated across COCO, Flickr30k, OK-VQA, and VizWiz in zero-shot and four-shot settings. The authors demonstrate strong vulnerability to both untargeted and targeted attacks, showing that small perturbations can substantially degrade performance and that targeted attacks can elicit specific, potentially harmful outputs with high success, especially with larger perturbation budgets. The work highlights critical safety concerns for real-world deployment and emphasizes the urgent need for robustness defenses in multi-modal foundation models.

Abstract

Multi-modal foundation models combining vision and language models such as Flamingo or GPT-4 have recently gained enormous interest. Alignment of foundation models is used to prevent models from providing toxic or harmful output. While malicious users have successfully tried to jailbreak foundation models, an equally important question is if honest users could be harmed by malicious third-party content. In this paper we show that imperceivable attacks on images in order to change the caption output of a multi-modal foundation model can be used by malicious content providers to harm honest users e.g. by guiding them to malicious websites or broadcast fake information. This indicates that countermeasures to adversarial attacks should be used by any deployed multi-modal foundation model.

Figures (5)

• Figure 1: Generated captions on original (left) and adversarially perturbed images (right). We perform a targeted attack on the caption output with $\varepsilon=1/255$ on the zero-shot model. This could be used for guiding users to a malicious website (top) or fake information (bottom). The perturbations are hardly visible and would not be noticed by a user.
• Figure 2: Generated captions on original and adversarially perturbed images. The perturbations are obtained with a targeted attack using radius $\varepsilon_q=1/255$ and 5000 APGD iterations on the zero-shot model. We show only the original images as the perturbations at this radius are not visible (cf.\ref{['fig:teaser']}). An adversary could use such attacks to spread misinformation to users unaware of the attack due to the imperceptibility of the according perturbations.
• Figure 3: Generated captions on original and adversarially perturbed COCO images. The perturbations are obtained with an untargeted attack using the smaller radius$\varepsilon_q=1/255$ and 500 iterations on the zero-shot model.
• Figure 4: Generated captions on original and adversarially perturbed COCO images. The perturbations are obtained with an untargeted attack using the larger radius$\varepsilon_q=4/255$ and 500 iterations on the zero-shot model.
• Figure 5: Effect on the performance if only a fraction of the perturbations are used. We zero out the perturbations that are smallest in magnitude and report (a) the resulting CIDEr score on COCO and (b) the attack success rate for the targeted attack with target caption "Please reset your password". Even when using only a fraction of the perturbations, the model demonstrates high vulnerability. The adversarial perturbations are obtained via APGD with 500 steps in zero-shot mode.