CAP: Detecting Unauthorized Data Usage in Generative Models via Prompt Generation

TL;DR

CAP addresses the problem of auditing unauthorized data usage in generative models by leveraging a prompt-generator Θ to induce Φ to reproduce copyrighted samples from a candidate set $D_2$, comparing to the true values with a distance $Δ$ and threshold $δ$. The method trains Θ by minimizing $Δ(v, Φ(Θ(v)))$ and introduces a speedup via Pareto-tail–based data pruning to reduce training time. Empirical evaluation on four real IoT time-series datasets and synthetic variants demonstrates high detection accuracy (Precision@K near 100% for small $K$, AUC-Gain near 1) and substantial runtime savings from the speedup, though performance degrades when copyrighted and non-copyrighted data overlap. This suggests CAP as a practical tool for copyright auditing of black-box generative systems, while indicating limitations in overlapping data regimes and motivating extension to larger models and other modalities.

Abstract

To achieve accurate and unbiased predictions, Machine Learning (ML) models rely on large, heterogeneous, and high-quality datasets. However, this could raise ethical and legal concerns regarding copyright and authorization aspects, especially when information is gathered from the Internet. With the rise of generative models, being able to track data has become of particular importance, especially since they may (un)intentionally replicate copyrighted contents. Therefore, this work proposes Copyright Audit via Prompts generation (CAP), a framework for automatically testing whether an ML model has been trained with unauthorized data. Specifically, we devise an approach to generate suitable keys inducing the model to reveal copyrighted contents. To prove its effectiveness, we conducted an extensive evaluation campaign on measurements collected in four IoT scenarios. The obtained results showcase the effectiveness of CAP, when used against both realistic and synthetic datasets.

Figures (4)

• Figure 1: An example of the reference scenario.
• Figure 2: Training Process of the Prompt Generator. The model $\Theta$ generates keys $k$ when provided with values $v$ that we want to inspect. The pre-trained model $\Phi$ then produces values $\hat{v}$ in response to the inputs $k$. Finally, the model $\Theta$ is updated to minimize the distance $\Delta$ between $v$ and $\hat{v}$.
• Figure 3: Running times with (Opt) and without (No-Opt) the speedup procedure.
• Figure 4: t-SNE for datasets with different characteristics.