MirrorMark: A Distortion-Free Multi-Bit Watermark for Large Language Models

TL;DR

MirrorMark advances LLM provenance attribution with a distortion-free, multi-bit watermark built on mod-1 mirroring of sampling randomness. It jointly integrates a context-anchored balanced scheduler (CABS) to robustly allocate tokens across message positions and three decoding/detection pathways (Gumbel-max, WeightedMean, Bayesian) to maximize per-token contrast while preserving the native output distribution. Theoretical EER analyses characterize detectability under both Gumbel-max and tournament schemes, and empirical results show strong detectability with minimal loss of text quality, including cross-language and robustness tests. This work broadens watermarking from binary signals to expressive, tamper-resilient multi-bit attribution, enabling richer provenance for AI-generated content with practical robustness.

Abstract

As large language models (LLMs) become integral to applications such as question answering and content creation, reliable content attribution has become increasingly important. Watermarking is a promising approach, but existing methods either provide only binary signals or distort the sampling distribution, degrading text quality; distortion-free approaches, in turn, often suffer from weak detectability or robustness. We propose MirrorMark, a multi-bit and distortion-free watermark for LLMs. By mirroring sampling randomness in a measure-preserving manner, MirrorMark embeds multi-bit messages without altering the token probability distribution, preserving text quality by design. To improve robustness, we introduce a context-based scheduler that balances token assignments across message positions while remaining resilient to insertions and deletions. We further provide a theoretical analysis of the equal error rate to interpret empirical performance. Experiments show that MirrorMark matches the text quality of non-watermarked generation while achieving substantially stronger detectability: with 54 bits embedded in 300 tokens, it improves bit accuracy by 8-12% and correctly identifies up to 11% more watermarked texts at 1% false positive rate.

Figures (16)

• Figure 1: Overview of CABS, where the number of positions $H$=4
• Figure 2: Asymptotic EER of MirrorMark based on Gumbel-max sampling (left) and tournament sampling (right). The token length is set to $T=200$. For tournament-based MirrorMark, we fix $m=1$.
• Figure 3: Empirical EER of MirrorMark on LLaMA-2-7B touvron2023llama with C4 prompts raffel2020exploring (left) and Gemma-7B-it gemmateam2024gemmaopenmodelsbased with ELI5 prompts fan-etal-2019-eli5 (right), with token length $T=200$. For tournament-based MirrorMark, $m=1$.
• Figure 4: Illustration of mod-1 mirroring under different message capacities, where 1-bit case is based on equation \ref{['mod-1mirroring-m1']}, while 2-bits case follows equation \ref{['eq:mod1-reflect-compact']}. In each subfigure, the first row partitions the original domain $u \in (0,1)$ into differently colored intervals to illustrate how portions of the domain are transformed. For a given message $M$, the star marker indicates the mirroring center $\psi_M$. Applying mod-1 mirroring around $\psi_M$ moves each original interval to a new location within $(0,1)$. Colors are used only to track this transformation: under a fixed $M$, all $u$ values that start in the same colored interval in the original row appear together in the interval with the same color after mirroring.
• Figure 5: Performance comparison between Gumbel-max-based MirrorMark and ThreeBricks, $H=1$.

Theorems & Definitions (4)

• Theorem 4.1
• Lemma 7.1
• Lemma 7.2
• Lemma 7.3