You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content

TL;DR

The paper investigates prompt learning with large language models to tackle toxic online content across toxicity classification, toxic span detection, and detoxification. By comparing manual and learnable prompts across five LM families and eight datasets, it demonstrates that prompt tuning can match or surpass task-specific baselines with far lower training costs and data requirements. Key findings include roughly a 10% F1 gain in toxicity classification, a 0.643 F1 score for toxic span detection (beating SPAN-BERT), and detoxification that substantially lowers toxicity while preserving semantics, though tradeoffs remain with semantic fidelity and cross-dataset transfer. The work highlights the practical potential of prompt tuning for scalable, environmentally friendlier toxic-content moderation and provides a foundation for integrating LLM-based prompts into annotation pipelines and moderation tooling.

Abstract

The spread of toxic content online is an important problem that has adverse effects on user experience online and in our society at large. Motivated by the importance and impact of the problem, research focuses on developing solutions to detect toxic content, usually leveraging machine learning (ML) models trained on human-annotated datasets. While these efforts are important, these models usually do not generalize well and they can not cope with new trends (e.g., the emergence of new toxic terms). Currently, we are witnessing a shift in the approach to tackling societal issues online, particularly leveraging large language models (LLMs) like GPT-3 or T5 that are trained on vast corpora and have strong generalizability. In this work, we investigate how we can use LLMs and prompt learning to tackle the problem of toxic content, particularly focusing on three tasks; 1) Toxicity Classification, 2) Toxic Span Detection, and 3) Detoxification. We perform an extensive evaluation over five model architectures and eight datasets demonstrating that LLMs with prompt learning can achieve similar or even better performance compared to models trained on these specific tasks. We find that prompt learning achieves around 10\% improvement in the toxicity classification task compared to the baselines, while for the toxic span detection task we find better performance to the best baseline (0.643 vs. 0.640 in terms of $F_1$-score). Finally, for the detoxification task, we find that prompt learning can successfully reduce the average toxicity score (from 0.775 to 0.213) while preserving semantic meaning.

Figures (7)

• Figure 1: $F_1$-score of Task 1 with different training steps.
• Figure 2: $F_1$-score of Task 2 (Toxic Span Detection) with different training epochs.
• Figure 3: Utility of Task 3 with different training epochs.
• Figure 4: Detoxification effectiveness of Task 3 with different training epochs.
• Figure 5: Accuracy of Task 1 with different training steps.