Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models

TL;DR

A no-go theorem is proved, which states that it is impossible to simultaneously maintain the highest watermark strength and the highest sampling efficiency, and two methods are proposed that maintain either the sampling efficiency or the watermark strength, but not both.

Abstract

Large language models are probabilistic models, and the process of generating content is essentially sampling from the output distribution of the language model. Existing watermarking techniques inject watermarks into the generated content without altering the output quality. On the other hand, existing acceleration techniques, specifically speculative sampling, leverage a draft model to speed up the sampling process while preserving the output distribution. However, there is no known method to simultaneously accelerate the sampling process and inject watermarks into the generated content. In this paper, we investigate this direction and find that the integration of watermarking and acceleration is non-trivial. We prove a no-go theorem, which states that it is impossible to simultaneously maintain the highest watermark strength and the highest sampling efficiency. Furthermore, we propose two methods that maintain either the sampling efficiency or the watermark strength, but not both. Our work provides a rigorous theoretical foundation for understanding the inherent trade-off between watermark strength and sampling efficiency in accelerating the generation of watermarked tokens for large language models. We also conduct numerical experiments to validate our theoretical findings and demonstrate the effectiveness of the proposed methods.

Figures (5)

• Figure 1: Taxonomy of watermarking and speculative sampling trade-offs in language models. The ideal case of maintaining both watermark strength and sampling efficiency is proven to be impossible by the no-go theorem. The proposed algorithms focus on maintaining either watermark strength or sampling efficiency.
• Figure 2: Comparison of different methods. The x-axis shows the Average Accepted Tokens Per Step (AATPS) as a measure of speculative sampling efficiency, while y-axis shows the Average Negative Log P-value Per Token (ANLPPT) as a measure of watermark strength. The P-value is computed based on either a likelihood-based test using the maximin-LLR score (left) or a likelihood-agnostic test using the U score (right). Watermarking is performed using either the DeltaGumbel reweight (top) or the Gamma reweight (bottom). Error bars represent $3\sigma$ confidence intervals.
• Figure 3: Text summarization task with LLaMa-13b model touvron2023llama as target model and LLaMa-68m model miao2023specinfer as reference model.
• Figure 4: Open-ended text generation task with LLaMa-7b model touvron2023llama as target model and LLaMa-68m model miao2023specinfer as reference model.
• Figure 5: Open-ended text generation task with LLaMa-13b model touvron2023llama as target model and LLaMa-68m model miao2023specinfer as reference model.

Theorems & Definitions (14)

• Theorem 1: No-go Theorem
• Remark 2: Condition for maintaining the watermark strength
• Lemma 3: Maintaining Sampling Efficiency Implies Unbiased Watermarked Draft Model
• Lemma 4: Maintaining Watermark Strength and Sampling Efficiency Implies Same Reweight Function
• proof : Proof of \ref{['thm:no_go']}
• Theorem 5: Maintaining Watermark Strength
• Theorem 6: Maintaining Sampling Efficiency
• Remark 7: Context code history
• proof : Proof of \ref{['lem:unbiased_draft']}
• proof : Proof of \ref{['lem:same_reweight']}