GPT-DETOX: An In-Context Learning-Based Paraphraser for Text Detoxification

TL;DR

The paper addresses the challenge of removing profanity and toxicity from text while preserving content without heavy fine-tuning. It proposes GPT-Detox, a prompt-based in-context learning framework that leverages zero-shot and few-shot prompting with two example-selection methods, WMES and CMES, and an ensemble approach (EICL) to maximize detoxification quality. On ParaDetox and APPDIA, zero-shot prompting demonstrates strong baseline performance, while CMES and especially EICL substantially improve the joint evaluation metric J = STA × SIM × FL, with S-score defined as the average of STA and SIM. The results, including human evaluations, indicate that instruction-tuned LLMs can effectively detoxify text in real time, offering a data-efficient alternative to fine-tuning with practical impact for moderating online content.

Abstract

Harmful and offensive communication or content is detrimental to social bonding and the mental state of users on social media platforms. Text detoxification is a crucial task in natural language processing (NLP), where the goal is removing profanity and toxicity from text while preserving its content. Supervised and unsupervised learning are common approaches for designing text detoxification solutions. However, these methods necessitate fine-tuning, leading to computational overhead. In this paper, we propose GPT-DETOX as a framework for prompt-based in-context learning for text detoxification using GPT-3.5 Turbo. We utilize zero-shot and few-shot prompting techniques for detoxifying input sentences. To generate few-shot prompts, we propose two methods: word-matching example selection (WMES) and context-matching example selection (CMES). We additionally take into account ensemble in-context learning (EICL) where the ensemble is shaped by base prompts from zero-shot and all few-shot settings. We use ParaDetox and APPDIA as benchmark detoxification datasets. Our experimental results show that the zero-shot solution achieves promising performance, while our best few-shot setting outperforms the state-of-the-art models on ParaDetox and shows comparable results on APPDIA. Our EICL solutions obtain the greatest performance, adding at least 10% improvement, against both datasets.