Watermarking Makes Language Models Radioactive

TL;DR

<3-5 sentence high-level summary> The paper investigates whether training a language model on text produced by a watermarked LLM leaves detectable traces, a phenomenon they term radioactivity. They develop statistically rigorous detectors for radioactivity across open/closed-model and supervised/unsupervised scenarios, leveraging watermarking signals and careful de-duplication to produce valid p-values. The experiments show strong, reliable detection even when only a small fraction of the fine-tuning data is watermarked, with open-model access providing the most powerful signals. The work highlights practical implications for IP protection and points to potential defenses, such as purification via successive fine-tuning, while emphasizing the need for careful handling of watermark design to sustain detectability. It demonstrates that watermarked training data can effectively contaminate downstream models, enabling high-confidence inferences about data provenance in real-world settings.

Abstract

We investigate the radioactivity of text generated by large language models (LLM), i.e. whether it is possible to detect that such synthetic input was used to train a subsequent LLM. Current methods like membership inference or active IP protection either work only in settings where the suspected text is known or do not provide reliable statistical guarantees. We discover that, on the contrary, it is possible to reliably determine if a language model was trained on synthetic data if that data is output by a watermarked LLM. Our new methods, specialized for radioactivity, detects with a provable confidence weak residuals of the watermark signal in the fine-tuned LLM. We link the radioactivity contamination level to the following properties: the watermark robustness, its proportion in the training set, and the fine-tuning process. For instance, if the suspect model is open-weight, we demonstrate that training on watermarked instructions can be detected with high confidence ($p$-value $< 10^{-5}$) even when as little as $5\%$ of training text is watermarked.

Figures (17)

• Figure 1: Bob fine-tunes his LLM on data with a fraction coming from Alice's LLM. This leaves traces in Bob's model that Alice can detect reliably, provided that her text was watermarked. Thus, a side effect of Alice's watermark, intended for machine-generated text detection, is to reveal what data Bob's model was fine-tuned on.
• Figure 1: Availability of radioactivity detection under the different settings. Open / closed-model refers to the availability of Bob's model, and supervised / unsupervised to Alice's knowledge of his data. Detection with watermarks is described in Sec. \ref{['sec:radioactivity_detection']}, and a baseline without WM relying on MIA in App. \ref{['par:mia_wm']}. Intellectual Property Protection (IPP) refers to zhao2023protecting; see App. \ref{['sec:ipp']}.
• Figure 2: Detection performance mainly depends on $\rho = |D^{\mathcal{A}}|/|D|$ and $d = |D^{\mathcal{A}}|/|\Tilde{D}^{\mathcal{A}}|$, where $D$ is the fine-tuning dataset used by Bob, $\Tilde{D}^{\mathcal{A}}$ are the outputs from Alice's model, and $D^{\mathcal{A}}$ the intersection of both.
• Figure 3: Radioactivity detection with closed or open model access (for simplicity, only kirchenbauer2023watermark is illustrated). (Left) New texts are generated from $\mathcal{B}$ using prompts from $\mathcal{A}$ and these texts are scored. The filter $\phi$ is used to focus the score computation on likely contaminated $k$-grams. (Right) Text generated by $\mathcal{A}$ are directly forwarded through $\mathcal{B}$, and the next-token predictions are scored using tokens from the input as the watermark window. In both cases, the tape ensures reliable $p$-values by de-duplicating scored tokens.
• Figure 4: Answers generated from Bob's model $\mathcal{B}$ (Llama-1), fine-tuned on instruction data generated by Alice's model $\mathcal{A}$ (Llama-2-chat) with different proportions $\rho$ of watermarked data. The quality of the instruction-tuning is not affected by the watermarking of the data (examples of training instruction/answer pairs are in Fig. \ref{['fig:example_self_instruct']}).

Theorems & Definitions (5)

• Definition 1: Text Radioactivity
• Definition 2: Model Radioactivity
• Proposition 1
• Proposition 2
• proof