Multi-Paradigm Collaborative Adversarial Attack Against Multi-Modal Large Language Models

TL;DR

A novel Multi-Paradigm Collaborative Attack (MPCAttack) framework to boost the transferability of adversarial examples against MLLMs, indicating that the solution consistently outperforms state-of-the-art methods in both targeted and untargeted attacks on open-source and closed-source MLLMs.

Abstract

The rapid progress of Multi-Modal Large Language Models (MLLMs) has significantly advanced downstream applications. However, this progress also exposes serious transferable adversarial vulnerabilities. In general, existing adversarial attacks against MLLMs typically rely on surrogate models trained within a single learning paradigm and perform independent optimisation in their respective feature spaces. This straightforward setting naturally restricts the richness of feature representations, delivering limits on the search space and thus impeding the diversity of adversarial perturbations. To address this, we propose a novel Multi-Paradigm Collaborative Attack (MPCAttack) framework to boost the transferability of adversarial examples against MLLMs. In principle, MPCAttack aggregates semantic representations, from both visual images and language texts, to facilitate joint adversarial optimisation on the aggregated features through a Multi-Paradigm Collaborative Optimisation (MPCO) strategy. By performing contrastive matching on multi-paradigm features, MPCO adaptively balances the importance of different paradigm representations and guides the global perturbation optimisation, effectively alleviating the representation bias. Extensive experimental results on multiple benchmarks demonstrate the superiority of MPCAttack, indicating that our solution consistently outperforms state-of-the-art methods in both targeted and untargeted attacks on open-source and closed-source MLLMs. The code is released at https://github.com/LiYuanBoJNU/MPCAttack.

Figures (8)

• Figure 1: A comparison of the proposed MPCAttack with typical attack solutions. (a) and (b) show previous approaches based on a single learning paradigm (e.g., CoA xie2025chain, FOA-Attack jia2025adversarial), while being limited by their transferability caused by restricted feature diversity and independent optimisation. (c) illustrates the core concept of MPCAttack, which integrates features from multiple learning paradigms and performs collaborative optimisation to enhance generalisation and transferability. Models and dashed ellipses in different colours represent the feature spaces of models delivered by different learning paradigms. (d) presents the attack success rates of different methods, where our MPCAttack obtains superior performance across various MLLMs consistently.
• Figure 2: Overview of the proposed MPCAttack: (a) Pipeline for MPCAttack in adversarial examples generation. (b) Pipeline for attacking MLLMs.
• Figure 3: Visualization of adversarial images and perturbations.
• Figure 4: Adversarial examples generated by MPCAttack, with responses from closed-source MLLMs.
• Figure 5: Ablation study for different learning paradigms and MPCO strategy.