Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models

TL;DR

The paper tackles scalable harmful-content moderation on social media by leveraging Large Language Models to perform few-shot in-context harm classification, including multimodal inputs from video thumbnails. It demonstrates that zero-shot LLMs can beat proprietary APIs and that few-shot in-context learning with a small exemplar set can approach domain-expert performance, with GPT-4o-Mini achieving near $80$% accuracy in multimodal setups. The study also analyzes the relative merits of caption-generation versus direct image input and conducts cross-dataset evaluations, showing generalization to hate speech and toxicity tasks. Overall, the work offers a practical, scalable path to dynamic moderation while acknowledging computational costs and ethical considerations.

Abstract

The prevalence of harmful content on social media platforms poses significant risks to users and society, necessitating more effective and scalable content moderation strategies. Current approaches rely on human moderators, supervised classifiers, and large volumes of training data, and often struggle with scalability, subjectivity, and the dynamic nature of harmful content (e.g., violent content, dangerous challenge trends, etc.). To bridge these gaps, we utilize Large Language Models (LLMs) to undertake few-shot dynamic content moderation via in-context learning. Through extensive experiments on multiple LLMs, we demonstrate that our few-shot approaches can outperform existing proprietary baselines (Perspective and OpenAI Moderation) as well as prior state-of-the-art few-shot learning methods, in identifying harm. We also incorporate visual information (video thumbnails) and assess if different multimodal techniques improve model performance. Our results underscore the significant benefits of employing LLM based methods for scalable and dynamic harmful content moderation online.

Figures (7)

• Figure 1: Our overall experimental framework utilizing LLMs for $k$-shot harmful content classification. The top approach showcases few-shot text-only in-context learning using the video's title (FS-ICL). The other two methods (FS-ICL-CG) are multimodal and augment the video title information with visual input (e.g. video thumbnail). In FS-ICL-CG we utilize a caption generation module (BLIP) to convert the visual input into text and feed these to a text-only LLM. In FS-ICL-DII we utilize fully multimodal LLMs and feed the visual input to the model directly. The output of each method is a classification of Harmful/Harmless for each given test input. Note that by setting $k=0$ we can obtain zero-shot learning (ZSL) variants for each of the aforementioned approaches.
• Figure 2: Accuracy (%) across different models for ZSL and FS-ICL (14-Shot; using BM25, Cosine, BSR selectors), proprietary baselines (Perspective and OpenAI Moderation APIs), crowdsourced annotators (Crowdsourced-Worst: minority label; Crowdsourced-Majority: majority label), and domain experts.
• Figure 3: Accuracy (%) of different LLMs for FS-ICL while varying number of shots and ICL selection methods, and open-source deep learning baselines (Matching Network and Prototypical Network) as well proprietary baselines. Among the selectors, BSR achieves the highest accuracy overall and across LLMs, GPT-4o-Mini achieves the highest performance. Interestingly, the number of shots has only a minor impact on the accuracy.
• Figure 4: Illustrative examples of few-shot ($k=8$) selection for two harm categories: (a) addictive gambling and (b) financial clickbait. Colors show how demonstrations cover salient aspects of the samples.
• Figure 5: Comparison of FS-ICL and FS-ICL-CG approaches of different models across different shot numbers and selection methods. FS-ICL-CG does not improve upon FS-ICL, indicating that inclusion of captions does not consistently enhance performance.