Evaluating Concept Filtering Defenses against Child Sexual Abuse Material Generation by Text-to-Image Models

TL;DR

The paper formalizes a security game around CSAM generation from text-to-image models and rigorously assesses concept-filtering defenses by evaluating automated child detection and adversarial use. It demonstrates that even strong automated detectors leave substantial residual risk, and that model-adaptation techniques like Fine-tuning and DreamBooth personalization can largely negate filtering, especially for open-weight models. The study also reveals unintended consequences on model generality and shows that even perfect filtering can be undermined by adaptation, underscoring the difficulty of achieving robust protections. Overall, the work highlights the substantial challenges in reliably preventing AIG-CSAM with current filtering approaches and calls for more effective detection, evaluation frameworks, and safeguards.

Abstract

We evaluate the effectiveness of child filtering to prevent the misuse of text-to-image (T2I) models to create child sexual abuse material (CSAM). First, we capture the complexity of preventing CSAM generation using a game-based security definition. Second, we show that current detection methods cannot remove all children from a dataset. Third, using an ethical proxy for CSAM (a child wearing glasses, hereafter, CWG), we show that even when only a small percentage of child images are left in the training dataset, there exist prompting strategies that generate CWG from a child-filtered T2I model using only a few more queries than when the model is trained on the unfiltered data. Fine-tuning the filtered model on child images further reduces the additional query overhead. We also show that reintroducing a concept is possible via fine-tuning even if filtering is perfect. Our results demonstrate that current filtering methods offer limited protection to closed-weight models and no protection to open-weight models, while reducing the generality of the model by hindering the generation of child-related concepts or changing their representation. We conclude by outlining challenges in conducting evaluations that establish robust evidence on the impact of AI safety mitigations for CSAM.

Figures (10)

• Figure 1: Examples of CWG images from each experiment with raters' median confidence that the image depicts a CWG. As expected due to its larger and human-centered training dataset, the LAION-Face model generates better images of children than the CC3M model.
• Figure 2: Raters' confidence that images in each experiment show a CWG.
• Figure 3: The estimated probability of obtaining at least one CWG image after the sequential number of queries $n$ for experiments on each dataset. Vertical lines denote $Q_{0.95}$.
• Figure 4: Violin plots: distribution of rater-reported ages for images classified as containing CWG in each experiment. White diamonds represent mean ages: CC3M (Unfiltered=7.6, Filtered-HP=15.0, Filtered-AP=14.6, Filtered-FT-HP=11.3 years) and LAION-Face (Unfiltered=7.4, Filtered-HP=15.4, Filtered-AP=13.9, Filtered-FT-HP=9.9 years).
• Figure 5: Age shift in images produced by CC3M (left) and LAION-face (right) models in response to heuristic prompts for children wearing glasses. In each row, we show images produced by unfiltered, filtered, and fine-tuned filtered models using the same prompt and seed, together with the minimum and maximum age rating for the youngest person in the image. See Appendix \ref{['appendix:additional_figures']} for details.

Theorems & Definitions (2)

• Definition 1: Concept
• Definition 2: Child