MC$^2$Mark: Distortion-Free Multi-Bit Watermarking for Long Messages

TL;DR

Experiments show that MC$^2$Mark improves detectability and robustness over prior multi-bit watermarking methods while preserving generation quality, achieving near-perfect accuracy for short messages and exceeding the second-best method by nearly 30% for long messages.

Abstract

Large language models now produce text indistinguishable from human writing, which increases the need for reliable provenance tracing. Multi-bit watermarking can embed identifiers into generated text, but existing methods struggle to keep both text quality and watermark strength while carrying long messages. We propose MC$^2$Mark, a distortion-free multi-bit watermarking framework designed for reliable embedding and decoding of long messages. Our key technical idea is Multi-Channel Colored Reweighting, which encodes bits through structured token reweighting while keeping the token distribution unbiased, together with Multi-Layer Sequential Reweighting to strengthen the watermark signal and an evidence-accumulation detector for message recovery. Experiments show that MC$^2$Mark improves detectability and robustness over prior multi-bit watermarking methods while preserving generation quality, achieving near-perfect accuracy for short messages and exceeding the second-best method by nearly 30% for long messages.

Figures (5)

• Figure 1: Overview of MC$^2$Mark framework. The generation process (left) employs Multi-Channel Colored Reweighting (MCCR) and Multi-Layer Sequential Reweighting ($m$ layers illustrated) to produce text without distortion. The detection process (right) utilizes evidence accumulation for accurate message extraction.
• Figure 2: Illustration of the Multi-Channel Colored Reweighting (MCCR) framework. The proposed method interprets the message bit vector $\mathbf{q}$ as color assignments for partitioned vocabulary subsets $\mathcal{V}^k$, where $q_i=1$ designates a green subset and $q_i=0$ designates a red subset. The diagram visualizes the reweighting process across the permutation space $\Pi$ to maintain distortion-free generation. The algorithm first attempts to uniformly amplify green subsets via a target scale $s^t$, computes the actual feasible green scale $s^a(\boldsymbol{\pi})$ based on the total green probability $\beta(\boldsymbol{\pi})$, and subsequently redistributes the remaining probability mass using the overflow scale $s^o(\boldsymbol{\pi})$.
• Figure 3: Detection accuracy as a function of text length for different message lengths $n \in \{16, 32, 64, 128\}$. Solid lines denote MC$^2$Mark, while dotted lines denote BiMark.
• Figure 4: Accuracy comparison of MC$^2$Mark, MC$^2$Mark without the mask bit, and the vanilla method under different message lengths. The vanilla method directly applies MCMark which amplifies only one channel in reweighting.
• Figure 5: Accuracy as a function of the number of layers $m$ for different message lengths $n \in \{16, 32, 64, 128, 256, 512\}$ on longform_qa dataset. The horizontal axis is shown on a logarithmic scale. Performance tends to peak at $m=20$.