Do Vision-Language Models Understand Visual Persuasiveness?

TL;DR

This work interrogates whether state-of-the-art vision–language models truly understand visual persuasiveness. By constructing a high-consensus image–message dataset and defining Visual Persuasive Factors (VPFs) across perceptual, compositional, and semantic levels, the authors dissect how humans and VLMs reason about persuasiveness. They find that high-level semantic alignment between the message and depicted objects best predicts human judgments, while VLMs exhibit a recall bias and rely poorly on low- and mid-level cues. Through cognitive steering and knowledge-injection experiments, they show that simple prompts are insufficient, but concise, object-grounded rationales can improve model precision and recall, highlighting the need for explicit grounding and reasoning about communicative intent. The findings suggest practical pathways to bolster persuasive understanding in VLMs and raise considerations for safe deployment in real-world media analysis and moderation.

Abstract

Recent advances in vision-language models (VLMs) have enabled impressive multi-modal reasoning and understanding. Yet, whether these models truly grasp visual persuasion-how visual cues shape human attitudes and decisions-remains unclear. To probe this question, we construct a high-consensus dataset for binary persuasiveness judgment and introduce the taxonomy of Visual Persuasive Factors (VPFs), encompassing low-level perceptual, mid-level compositional, and high-level semantic cues. We also explore cognitive steering and knowledge injection strategies for persuasion-relevant reasoning. Empirical analysis across VLMs reveals a recall-oriented bias-models over-predict high persuasiveness-and weak discriminative power for low/mid-level features. In contrast, high-level semantic alignment between message and object presence emerges as the strongest predictor of human judgment. Among intervention strategies, simple instruction or unguided reasoning scaffolds yield marginal or negative effects, whereas concise, object-grounded rationales significantly improve precision and F1 scores. These results indicate that VLMs core limitation lies not in recognizing persuasive objects but in linking them to communicative intent.

Figures (5)

• Figure 1: Overview of framework for Visual Persuasion Understanding (left) and the proposed Visual Persuasive Factors (right). The model (VLM) receives an image–message pair and judges persuasiveness, while human-identified factors—spanning low-perceptual, mid-compositional, high-semantic level, and high-semantic cues—enable deeper analysis of what drives persuasive reasoning.
• Figure 2: Overview of the high-level feature extraction pipeline. The framework detects whether semantically relevant key objects mentioned in the message are visually present in the image and identifies human presence through dedicated detectors.
• Figure 3: Illustrative low-level examples used in Appendix \ref{['app:low_example']}
• Figure 4: Illustrative examples used in Appendix \ref{['app:vpf_mid']}. Each example (top) is shown with its corresponding saliency map (bottom).
• Figure 5: High-level examples: each row shows a pair (original vs. object-detection overlay).