LR-DWM: Efficient Watermarking for Diffusion Language Models

TL;DR

This work tackles watermarking diffusion language models, where non-sequential denoising makes AR-based schemes ineffective. It introduces Left-Right Diffusion Watermarking (LR-DWM), a two-sided logit-biasing approach that leverages both left and right neighboring tokens via independent hash-based green sets, enabling schedule-agnostic watermarking with minimal overhead. Empirical results on LLaDA-8B-Instruct and DREAM-7B-Instruct show LR-DWM achieves high detectability with efficiency close to the non-watermarked baseline and competitive text quality compared to prior diffusion-watermarking methods, while maintaining robustness against common non-adaptive perturbations. The work also provides detailed analyses of trade-offs between detection and perplexity and discusses limitations such as vulnerability to paraphrasing and the need for sufficient text length. Overall, LR-DWM advances practical, scalable watermarking for diffusion language models by balancing detection performance, computational efficiency, and robustness in non-sequential decoding settings.

Abstract

Watermarking (WM) is a critical mechanism for detecting and attributing AI-generated content. Current WM methods for Large Language Models (LLMs) are predominantly tailored for autoregressive (AR) models: They rely on tokens being generated sequentially, and embed stable signals within the generated sequence based on the previously sampled text. Diffusion Language Models (DLMs) generate text via non-sequential iterative denoising, which requires significant modification to use WM methods designed for AR models. Recent work proposed to watermark DLMs by inverting the process when needed, but suffers significant computational or memory overhead. We introduce Left-Right Diffusion Watermarking (LR-DWM), a scheme that biases the generated token based on both left and right neighbors, when they are available. LR-DWM incurs minimal runtime and memory overhead, remaining close to the non-watermarked baseline DLM while enabling reliable statistical detection under standard evaluation settings. Our results demonstrate that DLMs can be watermarked efficiently, achieving high detectability with negligible computational and memory overhead.

Figures (4)

• Figure 1: Computational Efficiency. Peak GPU memory versus wall-clock generation time on a single NVIDIA H100. The non-watermarked baseline (green star) is shown for reference.
• Figure 2: Schematic of LR-DWM. For a target token $y_i$, the model’s logits distribution is modified using bidirectional local context. If available, the left neighbor $y_{i-1}$ and the right neighbor $y_{i+1}$ independently induce green-token constraints via hash functions with distinct keys $k_L$ and $k_R$. Each constraint applies an additive bias $\delta$ to its corresponding green set. Boundary tokens fall back to a single-sided constraint when one neighbor is missing.
• Figure 3: Quality-Detectability Trade-off. Detection rate at a FPR of 1% versus perplexity (PPL) across a range of bias strengths $\delta$. LR-DWM exhibits a competitive trade-off relative to DMARK and WM-DLM.
• Figure 4: Quality-Detectability Trade-off on DREAM. Detection rate versus perplexity (PPL) on the DREAM model for LR-DWM and prior watermarking methods, evaluated under stochastic decoding with low temperature. Each point corresponds to a different watermark strength $\delta$. LR-DWM exhibits a sharp detectability transition at lower PPL values, closely matching the trend observed on LLaDA and indicating consistent generalization across diffusion-based language models.