Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Watermark Stealing in Large Language Models

Nikola Jovanović, Robin Staab, Martin Vechev

TL;DR

This work is the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings, showing that for under $50 an attacker can both spoof and scrub state-of-the-art schemes previously considered safe, with average success rate of over 80%.

Abstract

LLM watermarking has attracted attention as a promising way to detect AI-generated content, with some works suggesting that current schemes may already be fit for deployment. In this work we dispute this claim, identifying watermark stealing (WS) as a fundamental vulnerability of these schemes. We show that querying the API of the watermarked LLM to approximately reverse-engineer a watermark enables practical spoofing attacks, as hypothesized in prior work, but also greatly boosts scrubbing attacks, which was previously unnoticed. We are the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings. We show that for under $50 an attacker can both spoof and scrub state-of-the-art schemes previously considered safe, with average success rate of over 80%. Our findings challenge common beliefs about LLM watermarking, stressing the need for more robust schemes. We make all our code and additional examples available at https://watermark-stealing.org.

Watermark Stealing in Large Language Models

TL;DR

This work is the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings, showing that for under $50 an attacker can both spoof and scrub state-of-the-art schemes previously considered safe, with average success rate of over 80%.

Abstract

LLM watermarking has attracted attention as a promising way to detect AI-generated content, with some works suggesting that current schemes may already be fit for deployment. In this work we dispute this claim, identifying watermark stealing (WS) as a fundamental vulnerability of these schemes. We show that querying the API of the watermarked LLM to approximately reverse-engineer a watermark enables practical spoofing attacks, as hypothesized in prior work, but also greatly boosts scrubbing attacks, which was previously unnoticed. We are the first to propose an automated WS algorithm and use it in the first comprehensive study of spoofing and scrubbing in realistic settings. We show that for under $50 an attacker can both spoof and scrub state-of-the-art schemes previously considered safe, with average success rate of over 80%. Our findings challenge common beliefs about LLM watermarking, stressing the need for more robust schemes. We make all our code and additional examples available at https://watermark-stealing.org.
Paper Structure (61 sections, 5 equations, 3 figures, 14 tables)

This paper contains 61 sections, 5 equations, 3 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Are LLM watermarks ready for deployment?
  3. This work: Watermark stealing in LLMs
  4. Spoofing attacks
  5. SOTA schemes can be spoofed
  6. Scrubbing attacks
  7. Stealing significantly boosts scrubbing
  8. Key contributions
  9. Background on LLM Watermarks
  10. KGW-Soft/Hard
  11. SelfHash and variants
  12. Spoofing-scrubbing tradeoff
  13. Unigram watermark
  14. The Watermark Stealing Threat Model
  15. Dimension: Detector access
  16. ...and 46 more sections

Figures (3)

  • Figure 1: Overview of watermark stealing and the downstream attacks it enables. (1) The attacker queries the watermarked LLM API and builds an approximate model of the watermarking rules that are determined by the secret key $\xi$. (2) The result of this can be used to spoof the watermark, i.e., generate watermarked text without knowing $\xi$. (3) Stealing also significantly boosts watermark scrubbing, i.e., the removal of the watermark from texts.
  • Figure 2: Top (\ref{['sec:flow']}): One iteration of our spoofing attack on KGW2-SelfHash. An open source model ( ) outputs a next-token distribution. The attacker modifies it based on the scoring function $s^\star$ previously learned via watermark stealing, and samples the token "best" that strengthens the watermark. A scrubbing attack would use a paraphraser as and set $\delta_{\text{att}}<0$ to weaken the watermark. Bottom (\ref{['sec:method']}): The components of the scoring function $s^\star$: the full context score, the optional partial context score from the dominant context token, the score based on the context-independent distribution.
  • Figure 3: Ablation and query cost study of spoofing of KGW2-SelfHash. $y$-axis shows the $\%$ of quality texts detected as watermarked under FPR of $10^{-6}$. No Partial Contexts refers to an ablation with $w_1=w_2=0$ (see \ref{['sec:method']}). Dashed ($\hbox{-}~\hbox{-}~\hbox{-}$) and full (---) lines correspond to average and top-1 results, respectively.