Enhancing Privacy in the Early Detection of Sexual Predators Through Federated Learning and Differential Privacy

TL;DR

This paper tackles the privacy challenges of early detection of online grooming by introducing a privacy-preserving eSPD pipeline that combines Federated Learning (FL) and Differential Privacy (DP). It systematically evaluates three DP-based training paradigms—metric DP, DP-SGD, and DP-FedAvg—and compares them against a standard FL baseline on the real-world PANC dataset, demonstrating that privacy can be maintained with only moderate utility loss. The results show FL can achieve competitive detection performance (e.g., F1 around 0.82, F-latency around 0.64) with low false positives, while DP variants trade some utility for stronger privacy guarantees. The work provides practical guidance for safeguarding child safety online without compromising user privacy and discusses ethical considerations, limitations, and future directions for privacy-preserving, real-time grooming detection.

Abstract

The increased screen time and isolation caused by the COVID-19 pandemic have led to a significant surge in cases of online grooming, which is the use of strategies by predators to lure children into sexual exploitation. Previous efforts to detect grooming in industry and academia have involved accessing and monitoring private conversations through centrally-trained models or sending private conversations to a global server. In this work, we implement a privacy-preserving pipeline for the early detection of sexual predators. We leverage federated learning and differential privacy in order to create safer online spaces for children while respecting their privacy. We investigate various privacy-preserving implementations and discuss their benefits and shortcomings. Our extensive evaluation using real-world data proves that privacy and utility can coexist with only a slight reduction in utility.

Figures (8)

• Figure 1: Illustration of the three DP implementations described in this work. Each step of the training process is represented: the learned BERT representation, the local training, and the global training. A lock representing DP is applied first to the embedding representation, then to the local training, and finally to the global training.
• Figure 2: eSPD: Training Phase
• Figure 3: eSPD: Inference Phase
• Figure 4: Illustration of the different privacy implementations: a lock representing DP is applied first to the embedding representation, then to the local training, and finally to the global training.
• Figure 5: Visualization of eSPD in which the risk is detected, a warning is raised after passing a threshold, and the user is notified as early as possible.