Efficient Toxicity Detection in Gaming Chats: A Comparative Study of Embeddings, Fine-Tuned Transformers and LLMs

TL;DR

The work systematically compares embeddings-based classifiers, zero-/few-shot LLM prompting, fine-tuned transformers, and retrieval-augmented generation for toxicity detection in online gaming chats. It demonstrates that fine-tuned DistilBERT offers the best accuracy-cost-speed balance, while LLMs (especially GPT-4) excel in contextual understanding but incur high latency and API costs; RAG achieves near-complete recall but trades off precision. A cascaded hybrid moderation architecture is proposed to maximize throughput and minimize human workload, with continuous learning to adapt to evolving language and policies. The study provides actionable guidance for deploying cost-effective, scalable, and ethically mindful content moderation in dynamic gaming ecosystems. Its findings have practical implications for platform developers, moderators, and policymakers seeking safer user experiences without sacrificing performance or privacy.

Abstract

This paper presents a comprehensive comparative analysis of Natural Language Processing (NLP) methods for automated toxicity detection in online gaming chats. Traditional machine learning models with embeddings, large language models (LLMs) with zero-shot and few-shot prompting, fine-tuned transformer models, and retrieval-augmented generation (RAG) approaches are evaluated. The evaluation framework assesses three critical dimensions: classification accuracy, processing speed, and computational costs. A hybrid moderation system architecture is proposed that optimizes human moderator workload through automated detection and incorporates continuous learning mechanisms. The experimental results demonstrate significant performance variations across methods, with fine-tuned DistilBERT achieving optimal accuracy-cost trade-offs. The findings provide empirical evidence for deploying cost-effective, efficient content moderation systems in dynamic online gaming environments.