Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information

Zheng Hui, Zhaoxiao Guo, Hang Zhao, Juanyong Duan, Congrui Huang

TL;DR

Harmful-content detection in NLP is hampered by data scarcity and inconsistent labeling across domains. The authors propose ToxiCraft, a framework that uses seed-guided synthesis with attribute extraction, context enhancement, and prompt-based refinements to generate diverse harmful-content samples via large language models, enabling training of smaller models with improved robustness. Across four datasets, ToxiCraft yields substantial improvements in Macro-F1, approaching or surpassing gold-label baselines, and demonstrates notable cross-dataset gains, particularly when transferring data between datasets. The work offers a practical, data-efficient path for robust harmful-content moderation, with ethical safeguards and clear directions for multilingual expansion and cost reduction.

Abstract

In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media. However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse. We propose Toxicraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses. With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.

ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information

TL;DR

Harmful-content detection in NLP is hampered by data scarcity and inconsistent labeling across domains. The authors propose ToxiCraft, a framework that uses seed-guided synthesis with attribute extraction, context enhancement, and prompt-based refinements to generate diverse harmful-content samples via large language models, enabling training of smaller models with improved robustness. Across four datasets, ToxiCraft yields substantial improvements in Macro-F1, approaching or surpassing gold-label baselines, and demonstrates notable cross-dataset gains, particularly when transferring data between datasets. The work offers a practical, data-efficient path for robust harmful-content moderation, with ethical safeguards and clear directions for multilingual expansion and cost reduction.

Abstract

In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media. However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse. We propose Toxicraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses. With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.
Paper Structure (28 sections, 8 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 28 sections, 8 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Harmful Information Detection
  4. Large Language Models
  5. Data Synthesis
  6. Methodology
  7. Problem Statement
  8. ToxiCraft Framework
  9. Experiment Setup
  10. Datasets and Processing
  11. Harmful Information Datasets
  12. Data Processing and Splits
  13. Baselines
  14. Implementation Details
  15. Experimental Results
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Harmful detection approaches
  • Figure 2: Our ToxiCraft Framework Diagram, CAE & TSR step is elaborating on transforming the initial data into multiple versions by altering the context while maintaining the core message or change the topic
  • Figure 3: Harmful detection approaches
  • Figure 4: Our ToxiCraft Framework example