PIXELMOD: Improving Soft Moderation of Visual Misleading Information on Twitter

TL;DR

PixelMod tackles the challenge of image-based misinformation by integrating fast perceptual hashes with OCR-derived contextual signals in a Milvus-backed vector index to enable scalable, context-aware soft moderation. The system generates dual syntactic embeddings ($pHash$ and $PDQHash$), retrieves visually similar images, and refines them using overlay-text OCR with a contextual similarity metric, achieving an optimal $\theta_{visual}$ of 90 for PDQHash and a textual threshold $\theta_{textual}$ of 0.05 using $Jaccard$ on $ngram=4$, yielding a peak $F1$ of $0.980$ ($\text{Prec}=0.990$, $\text{Rec}=0.979$). Evaluated on 2020 US election-related Twitter data, PixelMod detects misleading images with $0.99\%$ false positives and $2.06\%$ false negatives, substantially improving over baselines and highlighting substantial gaps in Twitter's own moderation. The work demonstrates how OCR-contextualized image similarity can extend soft moderation coverage at scale and offers an end-to-end, platform-agnostic blueprint for mitigating image-based misinformation in social media contexts, with implications for cross-platform deployment and policy-aligned moderation. $\,$

Abstract

Images are a powerful and immediate vehicle to carry misleading or outright false messages, yet identifying image-based misinformation at scale poses unique challenges. In this paper, we present PIXELMOD, a system that leverages perceptual hashes, vector databases, and optical character recognition (OCR) to efficiently identify images that are candidates to receive soft moderation labels on Twitter. We show that PIXELMOD outperforms existing image similarity approaches when applied to soft moderation, with negligible performance overhead. We then test PIXELMOD on a dataset of tweets surrounding the 2020 US Presidential Election, and find that it is able to identify visually misleading images that are candidates for soft moderation with 0.99% false detection and 2.06% false negatives.

Figures (7)

• Figure 1: Three tweets discussing a false electoral vote map showing Trump landslide victory based on false reports of seized election servers in Germany. Twitter added a warning label only to the first tweet.
• Figure 2: Overview of our image analysis pipeline.
• Figure 3: Example query image and results retrieved.
• Figure 4: F1 score of different grid search components.
• Figure 5: Latency of OCR by changing the percentage of text covering images in $\textbf{GT}_{viz}$.