Universal Harmful Information Synthesis via Model Crowdsourcing

TL;DR

The paper tackles the need for scalable, diverse harmful information data to robustly evaluate safeguards. It introduces SwarmLaunder, a strong-weak model collaboration framework that uses a Model Crowdsourcing Queue and Counterfactual Mapping to generate benign templates and toxify them across multiple LLMs. Adversarial Content Toxicifying then performs semantic decomposition, unit-level toxification, and coherent content reassembly, with Hallucination Evaluation to maintain quality. Empirically, SwarmLaunder achieves superior $SSR$, $Div$, $Tox$, and $Nat$ compared to baselines, and reveals notable differences between AI-generated and human-generated harmful content, underscoring the value of AI-driven, diverse data for advancing detectors and safeguards.

Abstract

To construct responsible and secure AI applications, harmful information data is widely utilized for adversarial testing and the development of safeguards. Existing studies mainly leverage Large Language Models (LLMs) to synthesize data to obtain high-quality task datasets at scale, thereby avoiding costly human annotation. However, limited by the safety alignment mechanisms of LLMs, the synthesis of harmful data still faces challenges in generation reliability and content diversity. In this study, we propose a novel harmful information synthesis framework, SwarmLaunder, which applies the model crowdsourcing strategy to generate diverse harmful data while maintaining a high success rate. Specifically, we generate abundant benign data as the based templates in a counterfactual manner. Subsequently, we decompose each based template into multiple semantic units and perform unit-by-unit toxification and final refinement through dynamic model switching, thus ensuring the success of synthesis. Experimental results demonstrate that SwarmLaunder achieves state-of-the-art performance in synthesizing different categories of harmful data with high scalability and diversity.

Figures (7)

• Figure 1: Illustration of different methods for harmful data construction. (a) Manual collection (left) curates and annotates real-world data from the online environment, but is limited by the scarcity and diversity of harmful data. (b) Data augmentation (right-top) generates abundant data by paraphrasing samples, but tends to produce homogeneous data with low toxic, e.g., offensive language. (c) Data synthesis (right-bottom) generates abundant and diverse data by utilizing LLMs' world knowledge. We decompose such harmful tasks and introduce the model crowdsourcing strategy to ensure the data diversity and success of generation for highly toxic data, e.g., phishing tweets.
• Figure 2: Comparison of existing harmful information detector performance on human/AI-generated content. Fig.\ref{['Fig.face']} demonstrates the cloud word of hate-speech targeting simulated user Johnny by AI generating. Fig.\ref{['Fig.radar']} illustrates the significant performance gap across multiple detectors, highlighting the challenges posed by AI-driven harmful speech campaigns.
• Figure 3: The overall structure of our SwarmLaunder. Specifically, Model Crowdsourcing Queue (§\ref{['4.0']}) is the core mechanism of SwarmLaunder to ensure the success of obtaining harmful content. In Counterfactual Mapping (§\ref{['4.1']}), SwarmLaunder first generates benign content with the desired structure and thematic elements. Then, in Adversarial Content Toxicifying (§\ref{['4.2']}), the benign content is segmented into smaller semantic units for targeted toxification. Multiple LLMs collaborate to transform these units into harmful content, with dynamic model switching when malfunctions occur. Finally, in Hallucination Evaluation (§\ref{['4.3']}), the generated harmful information is validated by its coherence and relevance.
• Figure 4: The overall process of Model Crowdsourcing Queue (MCQ).
• Figure 5: Illustration of detection performance (F1 and Accuracy) gap between Human (Top) and AI-generated (Bottom) harmful information, indicating the significant differences in harmful content between humans and AI.