Black-Box Detection of Language Model Watermarks

TL;DR

The paper investigates practical detectability of language-model watermarks under strict black-box access, challenging assumptions that watermarks remain undetectable in deployed settings. It develops principled statistical tests for three prominent watermark families—Red-Green, Fixed-Sampling, and Cache-Augmented—and demonstrates their effectiveness across seven schemes, five open-source models, and real APIs, while also providing parameter-estimation methods and robustness analyses. The results show that both distribution-modifying and distribution-preserving watermarks can be reliably detected with modest query costs, and they apply to deployed models like GPT-4, Claude, and Gemini in practical scenarios. The work highlights significant practical implications for model providers and regulatory oversight by revealing the relative ease of watermark detection in realistic black-box settings, while offering an accessible toolchain and open-source code for ongoing evaluation. Overall, the paper contributes a rigorous, empirically grounded framework for assessing watermark detectability and paves the way for more robust watermarking practices and security analyses in real-world LLM deployments.

Abstract

Watermarking has emerged as a promising way to detect LLM-generated text, by augmenting LLM generations with later detectable signals. Recent work has proposed multiple families of watermarking schemes, several of which focus on preserving the LLM distribution. This distribution-preservation property is motivated by the fact that it is a tractable proxy for retaining LLM capabilities, as well as the inherently implied undetectability of the watermark by downstream users. Yet, despite much discourse around undetectability, no prior work has investigated the practical detectability of any of the current watermarking schemes in a realistic black-box setting. In this work we tackle this for the first time, developing rigorous statistical tests to detect the presence, and estimate parameters, of all three popular watermarking scheme families, using only a limited number of black-box queries. We experimentally confirm the effectiveness of our methods on a range of schemes and a diverse set of open-source models. Further, we validate the feasibility of our tests on real-world APIs. Our findings indicate that current watermarking schemes are more detectable than previously believed.

Figures (6)

• Figure 1: Overview of our key contribution. Given black-box textual access to a language model, a client can query the model and run statistical tests to rigorously test for presence of a watermark. In this example, both the test for Cache-Augmented watermarks (\ref{['ssec:method:unbiased']}) and the test for Fixed-Sampling watermarks (\ref{['ssec:method:stanford']}) fail, while the test for Red-Green watermarks (\ref{['ssec:method:kgw']}) successfully detects the watermark.
• Figure 2: Left: distribution of bootstrapped p-values of the Red-Green test on Llama2-13B with $(\delta, \gamma)=(2, 0.25)$, for different sample sizes. We see reliable results for $100$ or more samples. Right: the diversity gap $n-R(n)$ on log scale in different settings. Linear behavior means that diversity scales exponentially with $t$, and we see that the assumption of $R(n)=n$ can be easily met in practice.
• Figure 3: Estimation of $\delta$ for different models using LeftHash with $\gamma = 0.25$. The number of samples used increases from left to right, with the leftmost plot assuming direct access to the log-probabilities. The estimation is done on the same data as the test. Error bars are given by the 95% bootstrapped confidence interval with respect to the sampling of the model outputs.
• Figure 4: Estimation of the context size $h$ in Red-Green watermarks with LeftHash $h=2$ (top) and LeftHash $h=3$ (bottom) on Llama2-7B. Each box corresponds to the distribution of $\hat{l}_{t_1,d,d',H}$. The green shading corresponds to the region where $\hat{h}_{t_1,d} \ge H$. A fixed $t_1$ is used across all plots.
• Figure 5: Left: power at 5% of the Fixed-Sampling test under the infinite diversity assumption. Right: power at 5% of the Cache-Augmented test assuming $f_1 = 0.5$. In both figures, the power is evaluated using 1000 repetitions of the test.