Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models
Minh-Hao Van, Xintao Wu
TL;DR
This work addresses the detection and mitigation of hateful content in multimodal memes using vision-language models. It introduces definition-guided prompting to enhance hateful-memes detection in a zero-/few-shot setting and presents UnHateMeme, a unified framework that transforms hateful memes into non-hateful, multimodal-coherent versions via GPT-4o-guided reasoning and multimodal generation. The approach achieves strong performance on hate-meme detection across seen and unseen data without fine-tuning and demonstrates substantial, human-validated improvements in mitigation quality, including high non-hatefulness and image-text coherence. By enabling end-to-end mitigation alongside detection, the paper contributes a practical pathway for safer online environments and highlights the capabilities and limitations of contemporary VLMs for handling harmful multimodal content, with future work including integration of image-generation models and deeper ethical considerations.
Abstract
The rapid evolution of social media has provided enhanced communication channels for individuals to create online content, enabling them to express their thoughts and opinions. Multimodal memes, often utilized for playful or humorous expressions with visual and textual elements, are sometimes misused to disseminate hate speech against individuals or groups. While the detection of hateful memes is well-researched, developing effective methods to transform hateful content in memes remains a significant challenge. Leveraging the powerful generation and reasoning capabilities of Vision-Language Models (VLMs), we address the tasks of detecting and mitigating hateful content. This paper presents two key contributions: first, a definition-guided prompting technique for detecting hateful memes, and second, a unified framework for mitigating hateful content in memes, named UnHateMeme, which works by replacing hateful textual and/or visual components. With our definition-guided prompts, VLMs achieve impressive performance on hateful memes detection task. Furthermore, our UnHateMeme framework, integrated with VLMs, demonstrates a strong capability to convert hateful memes into non-hateful forms that meet human-level criteria for hate speech and maintain multimodal coherence between image and text. Through empirical experiments, we show the effectiveness of state-of-the-art pretrained VLMs such as LLaVA, Gemini and GPT-4o on the proposed tasks, providing a comprehensive analysis of their respective strengths and limitations for these tasks. This paper aims to shed light on important applications of VLMs for ensuring safe and respectful online environments.