MMPersuade: A Dataset and Evaluation Framework for Multimodal Persuasion

TL;DR

The paper tackles how Large Vision-Language Models (LVLMs) behave as persuadees when exposed to multimodal persuasion. It introduces MMPersuade, a unified framework accompanied by a large-scale multimodal dataset (450 dialogues, 62,160 images, 4,756 videos) spanning Commercial, Subjective/Behavioral, and Adversarial contexts, and an evaluation protocol that measures both explicit agreement and implicit belief via the Persuasion Discounted Cumulative Gain (PDCG). Using six LVLMs, it shows that multimodal inputs amplify persuasion beyond text-only, that stronger initial preferences reduce susceptibility but are cushioned by visuals, and that strategy effectiveness varies by context (reciprocity/consistency for Commercial/Subjective; credibility/logic for Adversarial). The work provides a principled basis for designing LVLMs that are robust, preference-consistent, and ethically aligned when processing persuasive multimodal content, while also highlighting safety risks posed by visual misinformation. The dataset and artifacts promise reproducibility and further research into defenses, prompts, and policy interventions for multimodal persuasion.

Abstract

As Large Vision-Language Models (LVLMs) are increasingly deployed in domains such as shopping, health, and news, they are exposed to pervasive persuasive content. A critical question is how these models function as persuadees-how and why they can be influenced by persuasive multimodal inputs. Understanding both their susceptibility to persuasion and the effectiveness of different persuasive strategies is crucial, as overly persuadable models may adopt misleading beliefs, override user preferences, or generate unethical or unsafe outputs when exposed to manipulative messages. We introduce MMPersuade, a unified framework for systematically studying multimodal persuasion dynamics in LVLMs. MMPersuade contributes (i) a comprehensive multimodal dataset that pairs images and videos with established persuasion principles across commercial, subjective and behavioral, and adversarial contexts, and (ii) an evaluation framework that quantifies both persuasion effectiveness and model susceptibility via third-party agreement scoring and self-estimated token probabilities on conversation histories. Our study of six leading LVLMs as persuadees yields three key insights: (i) multimodal inputs substantially increase persuasion effectiveness-and model susceptibility-compared to text alone, especially in misinformation scenarios; (ii) stated prior preferences decrease susceptibility, yet multimodal information maintains its persuasive advantage; and (iii) different strategies vary in effectiveness across contexts, with reciprocity being most potent in commercial and subjective contexts, and credibility and logic prevailing in adversarial contexts. By jointly analyzing persuasion effectiveness and susceptibility, MMPersuade provides a principled foundation for developing models that are robust, preference-consistent, and ethically aligned when engaging with persuasive multimodal content.

Figures (40)

• Figure 1: Unified framework for studying multimodal persuasion. (Left) Persuasion contexts are organized into three contexts, with theory-grounded strategies. (Center) A dataset and dialogue setup where a persuader leverages the multimodal persuasion strategies dataset to compose multimodal persuasive messages and influence an LVLM persuadee's stance in multi-turn conversations, with shaded backgrounds indicating dataset-driven construction (left) versus LVLMs acting on behalf of human users (right). (Right) Persuasion effectiveness is evaluated by using two complementary stance evaluation methods, with metrics such as persuasion discounted cumulative gain measured across three dimensions: modality, stubbornness/preference, and strategy.
• Figure 2: Illustration of our dataset and evaluation framework. Each persuasive message appears in three settings, with required elements: textual response, image/video caption, textual description, or multimodal content. Varying the modality alters persuadee responses within the same turn, which are then evaluated using two complementary methods to capture stance shifts.
• Figure 3: PDCG scores across three contexts: Commercial (top), Subjective/Behavioral (middle), and Adversarial (bottom) -- for three persuader response types under linear and logarithmic discounting. Higher PDCG scores indicate earlier and more effective persuasion. Higher PDCG = earlier, more effective persuasion. Cool colors = greater susceptibility; warm = stronger resistance. Scoring: agreement (Commercial, Subjective/Behavioral); token probability (Adversarial).
• Figure 4: PDCG scores (logarithmic discount) for various models in Commercial Persuasion, evaluated via the agreement method. Preference strength levels range from 30 (weak) and 90 (strong).
• Figure 5: Persuasion dynamics under different system prompts in the Commercial Persuasion task. (Left) Convictions per round at preference level 50, comparing two evaluation methods: token probability and LLM agreement under a persona-role prompt. (Right) Differences in PDCG scores between multimodal and text-only inputs with no specified preference, across three system prompts: persona-role, assistant-role (without flexibility), and assistant-role (with flexibility).