Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Malicious Technical Ecosystem: Exposing Limitations in Technical Governance of AI-Generated Non-Consensual Intimate Images of Adults

Michelle L. Ding, Harini Suresh

TL;DR

The paper addresses the problem of AI-generated non-consensual intimate images (AIG-NCII) of adults and gaps in governance. It adopts a survivor-centered analysis that maps the Malicious Technical Ecosystem (MTE)—an open, decentralized stack of independently developed models, nudification tools, and supporting infrastructure—onto the NIST AI 100-4 synthetic-content governance framework. It identifies three key gaps: overreliance on transparency, conflation of CSAM with adult NCII, and governance focused on large corporate models rather than user-driven tools, arguing for survivor-centered prevention and framework reforms. By reframing AIG-NCII as a high-risk harm linked to the MTE, the work emphasizes regulatory shifts that address creation and dissemination, not just downstream removal, to reduce harms and improve practical protections for survivors.

Abstract

In this paper, we adopt a survivor-centered approach to locate and dissect the role of sociotechnical AI governance in preventing AI-Generated Non-Consensual Intimate Images (AIG-NCII) of adults, colloquially known as "deep fake pornography." We identify a "malicious technical ecosystem" or "MTE," comprising of open-source face-swapping models and nearly 200 "nudifying" software programs that allow non-technical users to create AIG-NCII within minutes. Then, using the National Institute of Standards and Technology (NIST) AI 100-4 report as a reflection of current synthetic content governance methods, we show how the current landscape of practices fails to effectively regulate the MTE for adult AIG-NCII, as well as flawed assumptions explaining these gaps.

The Malicious Technical Ecosystem: Exposing Limitations in Technical Governance of AI-Generated Non-Consensual Intimate Images of Adults

TL;DR

The paper addresses the problem of AI-generated non-consensual intimate images (AIG-NCII) of adults and gaps in governance. It adopts a survivor-centered analysis that maps the Malicious Technical Ecosystem (MTE)—an open, decentralized stack of independently developed models, nudification tools, and supporting infrastructure—onto the NIST AI 100-4 synthetic-content governance framework. It identifies three key gaps: overreliance on transparency, conflation of CSAM with adult NCII, and governance focused on large corporate models rather than user-driven tools, arguing for survivor-centered prevention and framework reforms. By reframing AIG-NCII as a high-risk harm linked to the MTE, the work emphasizes regulatory shifts that address creation and dissemination, not just downstream removal, to reduce harms and improve practical protections for survivors.

Abstract

In this paper, we adopt a survivor-centered approach to locate and dissect the role of sociotechnical AI governance in preventing AI-Generated Non-Consensual Intimate Images (AIG-NCII) of adults, colloquially known as "deep fake pornography." We identify a "malicious technical ecosystem" or "MTE," comprising of open-source face-swapping models and nearly 200 "nudifying" software programs that allow non-technical users to create AIG-NCII within minutes. Then, using the National Institute of Standards and Technology (NIST) AI 100-4 report as a reflection of current synthetic content governance methods, we show how the current landscape of practices fails to effectively regulate the MTE for adult AIG-NCII, as well as flawed assumptions explaining these gaps.

Paper Structure

This paper contains 11 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Response vs. Prevention: Locating the Role of Sociotechnical AI Governance in Mitigating AIG-NCII
  3. The Malicious Technical Ecosystem (MTE)
  4. Independently Developed Models
  5. User Facing Software and Independent "Deep Fake Creation Services"
  6. Supporting Technological Infrastructure
  7. Implications of the MTE: Limitations in Current Synthetic Content Governance Practices
  8. Limitation 1: Synthetic content governance measures focus on transparency, which is insufficient to address the harms of human-detectable and overtly watermarked adult AIG-NCII.
  9. Limitation 2: AIG-NCII specific measures conflate CSAM and adult NCII, proposing methods more effective in regulating CSAM than adult NCII.
  10. Limitation 3: Adult-specific AIG-NCII governance measures acknowledge the existence of the MTE but propose methods that will only regulate large corporate models.
  11. Looking Forward: Towards Survivor-Centered Prevention of Adult AIG-NCII

Figures (1)

  • Figure 1: Supply chain for AI Generated Non-Consensual Intimate Images that expands on the NIST AI 100-4 synthetic content pipeline of creation, distribution, and consumption. Yellow circles represent the malicious technical ecosystem (MTE). Training dataset, user input, and AIG-NCII are highlighted to represent artifacts within the pipeline and not any specific example.