A Marketplace for AI-Generated Adult Content and Deepfakes

TL;DR

This paper investigates governance and incentive structures on Civitai's bounty marketplace for AI-generated media, focusing on NSFW content and deepfakes. It employs a longitudinal, large-scale analysis of 4,847 bounties over fourteen months, using dual data collection passes, platform labels, external moderation, and GPT-based theme extraction, complemented by Lorenz curve and Gini analyses to assess participation concentration and interventions. The findings reveal a rising prevalence of NSFW bounties, pronounced deepfake activity targeting female public figures, and significant enforcement gaps in platform interventions, driven in part by LoRA-based tooling that enables steering model outputs beyond initial guardrails. These results highlight governance challenges for monetized, community-driven AI platforms and offer implications for platform designers, moderators, and policymakers regarding consent, enforcement, and the management of high-stakes content in AI-enabled ecosystems.

Abstract

Generative AI systems increasingly enable the production of highly realistic synthetic media. Civitai, a popular community-driven platform for AI-generated content, operates a monetized feature called Bounties, which allows users to commission the generation of content in exchange for payment. To examine how this mechanism is used and what content it incentivizes, we conduct a longitudinal analysis of all publicly available bounty requests collected over a 14-month period following the platform's launch. We find that the bounty marketplace is dominated by tools that let users steer AI models toward content they were not trained to generate. At the same time, requests for content that is "Not Safe For Work" are widespread and have increased steadily over time, now comprising a majority of all bounties. Participation in bounty creation is uneven, with 20% of requesters accounting for roughly half of requests. Requests for "deepfake" - media depicting identifiable real individuals - exhibit a higher concentration than other types of bounties. A nontrivial subset of these requests involves explicit deepfakes despite platform policies prohibiting such content. These bounties disproportionately target female celebrities, revealing a pronounced gender asymmetry in social harm. Together, these findings show how monetized, community-driven generative AI platforms can produce gendered harms, raising questions about consent, governance, and enforcement.

Figures (5)

• Figure 1: Example Civitai bounties. (a) A bounty with non-blurred images. (b) A bounty having blurred images of different ratings. The bounty poster's names (under 'Supporters') are hidden for privacy reasons.
• Figure 2: Civitai's bounty marketplace shows a substantial and growing proportion of NSFW requests. (a) Distribution of bounty request types. (b) Distribution of bounty themes. (c) Proportions of SFW (blue) and NSFW (red) bounties across different content types, highlighting that certain categories, videos in particular, have a higher proportion of NSFW requests. (d) Proportions of SFW (blue) and NSFW (red) bounties over time, showing a decline in SFW and a rise in NSFW bounties. Each point represents a weekly proportion, with ordinary least squares regression fit lines and bootstrapped 95% confidence intervals ($n=1{,}000$ samples).
• Figure 3: Deepfake details among Civitai bounties. (a) Distribution of targeted genders in the deepfake requests. (b) Distribution of professions as found in the official websites of the real persons targeted in the deepfake bounty requests. The "self/spouse" tag refers to bounties claiming to target the supporter or their spouse.
• Figure 4: Lorenz curves and Gini coefficients showing user concentration across bounty categories. The dashed line denotes perfect equality; greater distance from the dashed line indicates that fewer users account for a larger share of bounty requests.
• Figure 5: Proportions of SFW and NSFW deepfake bounties that display the platform's informational alert. Civitai's deepfake intervention is absent from a substantial fraction of deepfake bounty requests, particularly for NSFW content.