Deepfake Pornography is Resilient to Regulatory and Platform Shocks

TL;DR

The paper investigates whether federal regulation via the TAKE IT DOWN Act and the shutdown of a major SNCEI hub reduce synthetic non-consensual explicit imagery, or merely shift activity across ecosystems. Using a synthetic-control design across three platforms and weekly metrics of SNCEI posts, new contributors, and requests, the authors find no durable suppression; SNCEI activity persists and often grows, with evidence of cross-platform displacement. The responses are heterogeneous in timing and magnitude, including anticipation effects on some sites. These findings imply that regulatory and platform shocks may reallocate SNCEI production and sharing rather than meaningfully decrease its prevalence, highlighting the need for more robust enforcement and deeper understanding of migration dynamics in online abuse ecosystems.

Abstract

Generative artificial intelligence tools have made it easier to create realistic, synthetic non-consensual explicit imagery (popularly known as deepfake pornography; hereinafter SNCEI) of people. Once created, this SNCEI is often shared on various websites, causing significant harm to victims. This emerging form of sexual abuse was recently criminalized in the US at the federal level by S.146, the TAKE IT DOWN Act. A week after the bill's passage became effectively imminent, the MrDeepfakes website -- one of the most notorious facilitators of SNCEI creation and dissemination -- shut down. Here, we explore the impact of the bill's passage and the subsequent shutdown as a compound intervention on the dissemination of SNCEI. We select three online forums where sexually explicit content is shared, each containing dedicated subforums to organize various types of sexually explicit content. By leveraging each forum's design, we compare activity in subforums dedicated to SNCEI with that in other pornographic genres using a synthetic control, quasi-experimental approach. Across websites, we observed an increase in the sharing and requests for SNCEI, and, in some cases, in new contributors. These results indicate that the compound intervention did not suppress SNCEI activity overall but instead coincided with its redistribution across platforms, with substantial heterogeneity in timing and magnitude. Together, our findings suggest that deplatforming and regulatory signals alone may shift where and when SNCEI is produced and shared, rather than reducing its prevalence.

Figures (4)

• Figure 1: Motivation. We study the impact of the TAKE IT DOWN Act and the deplatforming of MrDeepFakes on the production of synthetic non-consensual explicit imagery (SNCEI) in three other websites known to host such content. Within these websites, we use the volume of non-SNCEI-related content (dotted blue in the figure) to derive a counterfactual expected volume of SNCEI had the shocks not occurred (the synthetic control; solid blue). We contrast it with the observed sharing of SNCEI content (solid black), finding that SNCEI sharing was resilient to this compound intervention, with significant spillovers to other platforms.
• Figure 2: New posts and newly active contributors for each treated unit across the three websites. Vertical lines represent the passing of the TAKE IT DOWN Act at the US Senate, UK OSA, US House of Representatives (main intervention*), and the MrDeepfakes shutdown, in order. Numerical results are shown in Table \ref{['tab:synth-results']}. Results for "Celebrities" are shown in Figure \ref{['fig:additional']}.
• Figure 3: Placebo test results. Solid lines show the difference between each treated unit and its synthetic control. The black line denotes the treated unit; gray lines are possible outcomes for control units. Results for “Celebrities” appear in Figure \ref{['fig:additional']}.
• Figure 4: Continued from Figures \ref{['fig:main-synth']} and \ref{['fig:placebo-synth']}, graphs corresponding to "Celebrities" posts on Website B.