Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improve the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models

Weiqing He, Xiang Li, Li Shen, Weijie Su, Qi Long

TL;DR

This work reexamines the perceived impossibility of simultaneously maximizing watermark strength and speculative-sampling efficiency in language models. It introduces a continuous watermark-strength measure based on $WS = \mathbb{E}_{\zeta}[D_{KL}(P_{\zeta} \| P)]$, showing it is maximized when tokens are deterministic functions of pseudorandomness and connecting it to detection difficulty via sample complexity. The authors formalize the trade-off as a Pareto frontier between watermark strength and sampling efficiency, derive explicit curves for prominent schemes, and then propose a principled pseudorandom-acceptance mechanism that achieves maximal watermark strength without sacrificing efficiency, with empirical validation of improved detectability. Collectively, the results offer a concrete, practical pathway to deploy watermarking with speculative sampling in real-world LLM systems while preserving provenance signals and throughput.

Abstract

Watermarking is a principled approach for tracing the provenance of large language model (LLM) outputs, but its deployment in practice is hindered by inference inefficiency. Speculative sampling accelerates inference, with efficiency improving as the acceptance rate between draft and target models increases. Yet recent work reveals a fundamental trade-off: higher watermark strength reduces acceptance, preventing their simultaneous achievement. We revisit this trade-off and show it is not absolute. We introduce a quantitative measure of watermark strength that governs statistical detectability and is maximized when tokens are deterministic functions of pseudorandom numbers. Using this measure, we fully characterize the trade-off as a constrained optimization problem and derive explicit Pareto curves for two existing watermarking schemes. Finally, we introduce a principled mechanism that injects pseudorandomness into draft-token acceptance, ensuring maximal watermark strength while maintaining speculative sampling efficiency. Experiments further show that this approach improves detectability without sacrificing efficiency. Our findings uncover a principle that unites speculative sampling and watermarking, paving the way for their efficient and practical deployment.

Improve the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models

TL;DR

This work reexamines the perceived impossibility of simultaneously maximizing watermark strength and speculative-sampling efficiency in language models. It introduces a continuous watermark-strength measure based on , showing it is maximized when tokens are deterministic functions of pseudorandomness and connecting it to detection difficulty via sample complexity. The authors formalize the trade-off as a Pareto frontier between watermark strength and sampling efficiency, derive explicit curves for prominent schemes, and then propose a principled pseudorandom-acceptance mechanism that achieves maximal watermark strength without sacrificing efficiency, with empirical validation of improved detectability. Collectively, the results offer a concrete, practical pathway to deploy watermarking with speculative sampling in real-world LLM systems while preserving provenance signals and throughput.

Abstract

Watermarking is a principled approach for tracing the provenance of large language model (LLM) outputs, but its deployment in practice is hindered by inference inefficiency. Speculative sampling accelerates inference, with efficiency improving as the acceptance rate between draft and target models increases. Yet recent work reveals a fundamental trade-off: higher watermark strength reduces acceptance, preventing their simultaneous achievement. We revisit this trade-off and show it is not absolute. We introduce a quantitative measure of watermark strength that governs statistical detectability and is maximized when tokens are deterministic functions of pseudorandom numbers. Using this measure, we fully characterize the trade-off as a constrained optimization problem and derive explicit Pareto curves for two existing watermarking schemes. Finally, we introduce a principled mechanism that injects pseudorandomness into draft-token acceptance, ensuring maximal watermark strength while maintaining speculative sampling efficiency. Experiments further show that this approach improves detectability without sacrificing efficiency. Our findings uncover a principle that unites speculative sampling and watermarking, paving the way for their efficient and practical deployment.
Paper Structure (48 sections, 6 theorems, 63 equations, 15 figures, 2 tables, 1 algorithm)

This paper contains 48 sections, 6 theorems, 63 equations, 15 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Paper organization.
  4. Preliminaries
  5. Gumbel-max watermark.
  6. SynthID watermark.
  7. Speculative sampling.
  8. An "inevitable" trade-off.
  9. Complete the Trade-off Curve
  10. Quantification of Watermark Strength
  11. Interpretation in terms of sample complexity.
  12. Maximum watermark strength.
  13. Trade-off Curves and Examples
  14. Formulation of trade-off curves.
  15. An example trade-off curve.
  16. ...and 33 more sections

Key Result

Theorem 3.1

Let $\alpha \in (0,1)$ and ${w}_{1:n} = (w_1, \ldots, w_n)$. Consider the hypothesis testing problem based on $n$ independent samples: where each $\zeta_t$ is i.i.d., and the log-likelihood ratios $Z_t := \log \frac{\bm{P}_{t,\zeta_t}({w}_t)}{\bm{P}_t({w}_t)}$ are independent, uniformly bounded, and admit a common neighborhood around zero where their moment generating functions are finite. Assume

Figures (15)

  • Figure 1: Trade-off curves between watermark strength and sampling efficiency for simulated $(\bm{Q}, \bm{P})$ pairs. Left: Curves for the linearly watermarked classes (\ref{['trade-off-linear']}). Right: Curves for other classes, including Hu's class hu2024inevitable and Google's class Dathathri2024. Orange and blue denote Gumbel-max and SynthID, respectively. Here, solid, dashed, and dotted lines correspond to different classes. Markers indicate the boundary point of each curve.
  • Figure 2: Left: Average Accepted Tokens Per Step (AATPS) of Alg. \ref{['alg:watermarked-spec']} applied to the Gumbel-max and SynthID watermarks, compared with Standard Speculative Sampling (Std. SpecSampl). Error bars mark the 95% confidence intervals. Middle and Right: Watermark detectability (TPR at FPR = 1%) for Alg. \ref{['alg:watermarked-spec']} on the Gumbel-max (middle) and SynthID (right). Orange curves show our method, blue curves show the prior-based method, and black curves represent the ideal detector (Oracle) that always selects the correct test statistic. Shaded regions indicate the 95% confidence intervals.
  • Figure 3: Experimental results for Gemma models on the ELI5 dataset. Left: Average Accepted Tokens Per Step (AATPS) of Alg. \ref{['alg:watermarked-spec']} applied to the Gumbel-max and SynthID watermarks, compared with Standard Speculative Sampling (Std. SpecSampl). Error bars mark the 95% confidence intervals. Middle and Right: Watermark detectability (TPR at FPR = 1%) for Alg. \ref{['alg:watermarked-spec']} on the Gumbel-max (middle) and SynthID (right). Orange curves show our method, blue curves show the prior-based method, and black curves represent the ideal detector (Oracle) that always selects the correct test statistic. Shaded regions indicate the 95% confidence intervals.
  • Figure 5: Experimental results for Llama models on the C4 dataset. Left: Average Accepted Tokens Per Step (AATPS) of Alg. \ref{['alg:watermarked-spec']} applied to the Gumbel-max and SynthID watermarks, compared with Standard Speculative Sampling (Std. SpecSampl). Error bars mark the 95% confidence intervals. Middle and Right: Watermark detectability (TPR at FPR = 1%) for Alg. \ref{['alg:watermarked-spec']} on the Gumbel-max (middle) and SynthID (right). Orange curves show our method, blue curves show the prior-based method, and black curves represent the ideal detector (Oracle) that always selects the correct test statistic. Shaded regions indicate the 95% confidence intervals.
  • Figure 6: Experimental results for Gemma models on the C4 dataset. Left: Average Accepted Tokens Per Step (AATPS) of Alg. \ref{['alg:watermarked-spec']} applied to the Gumbel-max and SynthID watermarks, compared with Standard Speculative Sampling (Std. SpecSampl). Error bars mark the 95% confidence intervals. Middle and Right: Watermark detectability (TPR at FPR = 1%) for Alg. \ref{['alg:watermarked-spec']} on the Gumbel-max (middle) and SynthID (right). Orange curves show our method, blue curves show the prior-based method, and black curves represent the ideal detector (Oracle) that always selects the correct test statistic. Shaded regions indicate the 95% confidence intervals.
  • ...and 10 more figures

Theorems & Definitions (17)

  • Definition 2.1: Sampling efficiency
  • Definition 3.1
  • Remark 3.1
  • Theorem 3.1: Sample complexity via $p$-value decay
  • Theorem 3.2: Maximum watermark strength
  • Theorem 3.3
  • Definition 3.2: Trade-off curve
  • Lemma 3.1: Speculative sampling is optimal
  • Remark 3.2
  • Theorem 4.1
  • ...and 7 more