OSPC: Detecting Harmful Memes with Large Language Model as a Catalyst

TL;DR

OSPC tackles harmful meme detection in Singapore's multilingual online ecosystem by fusing image captioning, OCR, and LLM reasoning to interpret multimodal memes across English, Chinese, Malay, and Tamil. The system combines BLIP for image descriptions with multilingual OCR and a Qwen-based harm classifier, supplemented by Tamil translation to bridge low-resource gaps and GPT-4V-guided data augmentation for fine-tuning via QLoRA. Key contributions include a three-stage multimodal framework, synthetic OCR data generation, and cross-language handling with targeted prompts and logits-based harm probability estimation. On the AI Singapore Online Safety Prize Challenge, the approach achieved top-1 with AUROC 0.7749 and accuracy 0.7087, outperforming prior benchmarks such as FLAVA and VisualBERT, demonstrating practical potential for scalable multilingual online safety tooling.

Abstract

Memes, which rapidly disseminate personal opinions and positions across the internet, also pose significant challenges in propagating social bias and prejudice. This study presents a novel approach to detecting harmful memes, particularly within the multicultural and multilingual context of Singapore. Our methodology integrates image captioning, Optical Character Recognition (OCR), and Large Language Model (LLM) analysis to comprehensively understand and classify harmful memes. Utilizing the BLIP model for image captioning, PP-OCR and TrOCR for text recognition across multiple languages, and the Qwen LLM for nuanced language understanding, our system is capable of identifying harmful content in memes created in English, Chinese, Malay, and Tamil. To enhance the system's performance, we fine-tuned our approach by leveraging additional data labeled using GPT-4V, aiming to distill the understanding capability of GPT-4V for harmful memes to our system. Our framework achieves top-1 at the public leaderboard of the Online Safety Prize Challenge hosted by AI Singapore, with the AUROC as 0.7749 and accuracy as 0.7087, significantly ahead of the other teams. Notably, our approach outperforms previous benchmarks, with FLAVA achieving an AUROC of 0.5695 and VisualBERT an AUROC of 0.5561.

Figures (2)

• Figure 1: The system pipeline for meme analysis.
• Figure 2: The construction method of OCR datasets.