MEMEWEAVER: Inter-Meme Graph Reasoning for Sexism and Misogyny Detection

TL;DR

MemeWeaver tackles the challenge of detecting sexism and misogyny in memes by modeling batch-level inter-meme relations with a novel Inter-Meme Graph Reasoning module and by fusing text and image information through learnable multimodal fusion. It verbalizes meme text via optional LLM captions, encodes modalities with CLIP-based backbones, and enriches meme representations through a fully learnable batch graph before classification. The approach achieves state-of-the-art performance on the MAMI and EXIST benchmarks, with faster training convergence and interpretable graph structures that reveal relational patterns in online hate. These results highlight the utility of batch-level relational reasoning for robust, context-aware multimodal hate-speech detection and point to future work on broader tasks and multilingual settings.

Abstract

Women are twice as likely as men to face online harassment due to their gender. Despite recent advances in multimodal content moderation, most approaches still overlook the social dynamics behind this phenomenon, where perpetrators reinforce prejudices and group identity within like-minded communities. Graph-based methods offer a promising way to capture such interactions, yet existing solutions remain limited by heuristic graph construction, shallow modality fusion, and instance-level reasoning. In this work, we present MemeWeaver, an end-to-end trainable multimodal framework for detecting sexism and misogyny through a novel inter-meme graph reasoning mechanism. We systematically evaluate multiple visual--textual fusion strategies and show that our approach consistently outperforms state-of-the-art baselines on the MAMI and EXIST benchmarks, while achieving faster training convergence. Further analyses reveal that the learned graph structure captures semantically meaningful patterns, offering valuable insights into the relational nature of online hate.

Figures (11)

• Figure 1: Overview of MemeWeaver, our graph-based framework for end-to-end sexism/misogyny detection in memes. Motivated by the social dynamics of online hate, it models batches of memes (3 shown) as inter-connected nodes for more effective learning.
• Figure 2: Architecture of MemeWeaver. Each meme undergoes text extraction (OCR, optionally enriched with LLM-generated captions) and is encoded by separate text and image encoders. The embeddings are fused, refined via inter-meme graph reasoning within each batch, and classified. Example shown with batch size $m=3$.
• Figure 3: Prompts used for LLM-based meme captioning. The HATE_TYPE placeholder was set to misogynistic or sexist depending on the downstream dataset.
• Figure 4: Language distribution in EXIST. Ratio of sexist vs. non-sexist memes across splits.
• Figure 5: Embedding space analysis. 2D visualizations of MAMI and EXIST test set embeddings $\mathbf{f}'$ and $\mathbf{f}$, generated with and without MemeWeaver's IMGR, respectively. Decision boundaries from a linear classifier show higher accuracy on $\mathbf{f}'$ (see scores below plots), underscoring the benefit of IMGR's representations.