Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WorldCup Sampling for Multi-bit LLM Watermarking

Yidan Wang, Yubing Ren, Yanan Cao, Li Guo

TL;DR

WorldCup tackles reliable provenance of LLM-generated text by directly embedding multi-bit messages into token selection through a hierarchical, tournament-based sampling framework. It introduces complementary g-value functions and entropy-aware modulation to balance watermark capacity with text quality, and a confidence-aware decoding scheme to robustly recover embedded bits. Empirical results across multiple models and tasks show WorldCup outperforms prior baselines in decoding accuracy, detectability, robustness to attacks, and efficiency, while preserving fluency. This approach provides a principled, scalable foundation for scalable LLM watermarking and practical content attribution.

Abstract

As large language models (LLMs) generate increasingly human-like text, watermarking offers a promising solution for reliable attribution beyond mere detection. While multi-bit watermarking enables richer provenance encoding, existing methods largely extend zero-bit schemes through seed-driven steering, leading to indirect information flow, limited effective capacity, and suboptimal decoding. In this paper, we propose WorldCup, a multi-bit watermarking framework for LLMs that treats sampling as a natural communication channel and embeds message bits directly into token selection via a hierarchical competition mechanism guided by complementary signals. Moreover, WorldCup further adopts entropy-aware modulation to preserve generation quality and supports robust message recovery through confidence-aware decoding. Comprehensive experiments show that WorldCup achieves a strong balance across capacity, detectability, robustness, text quality, and decoding efficiency, consistently outperforming prior baselines and laying a solid foundation for future LLM watermarking studies.

WorldCup Sampling for Multi-bit LLM Watermarking

TL;DR

WorldCup tackles reliable provenance of LLM-generated text by directly embedding multi-bit messages into token selection through a hierarchical, tournament-based sampling framework. It introduces complementary g-value functions and entropy-aware modulation to balance watermark capacity with text quality, and a confidence-aware decoding scheme to robustly recover embedded bits. Empirical results across multiple models and tasks show WorldCup outperforms prior baselines in decoding accuracy, detectability, robustness to attacks, and efficiency, while preserving fluency. This approach provides a principled, scalable foundation for scalable LLM watermarking and practical content attribution.

Abstract

As large language models (LLMs) generate increasingly human-like text, watermarking offers a promising solution for reliable attribution beyond mere detection. While multi-bit watermarking enables richer provenance encoding, existing methods largely extend zero-bit schemes through seed-driven steering, leading to indirect information flow, limited effective capacity, and suboptimal decoding. In this paper, we propose WorldCup, a multi-bit watermarking framework for LLMs that treats sampling as a natural communication channel and embeds message bits directly into token selection via a hierarchical competition mechanism guided by complementary signals. Moreover, WorldCup further adopts entropy-aware modulation to preserve generation quality and supports robust message recovery through confidence-aware decoding. Comprehensive experiments show that WorldCup achieves a strong balance across capacity, detectability, robustness, text quality, and decoding efficiency, consistently outperforming prior baselines and laying a solid foundation for future LLM watermarking studies.
Paper Structure (71 sections, 4 theorems, 32 equations, 12 figures, 15 tables, 2 algorithms)

This paper contains 71 sections, 4 theorems, 32 equations, 12 figures, 15 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations.
  4. SynthID-Text Zero-bit Watermarking
  5. Watermark Embedding.
  6. Watermark Detection.
  7. MPAC Multi-bit Watermarking
  8. Message Encoding.
  9. Message Decoding.
  10. Methodology
  11. Overview.
  12. Binary WorldCup Watermarking
  13. Complementary $g$-value Functions
  14. Generalized WorldCup Watermarking
  15. Confidence-aware Message Decoding
  16. ...and 56 more sections

Key Result

Proposition 3.5

Let $g_\ell(x,r)$ be a Bernoulli $g$-value and $\bar{g}_\ell(x,r)$ its complementary counterpart. Encoding bits $0$ and $1$ by selecting between $g_\ell$ and $\bar{g}_\ell$ yields two embedding distributions whose statistical discriminability is maximized.

Figures (12)

  • Figure 1: An overview of our multi-bit watermarking framework WorldCup for large language models.
  • Figure 2: Binary complementary $g$-values vs. random $g$-values.
  • Figure 3: The comparison of multi-bit message decoding.
  • Figure 4: The AUROC curves under different attacks on LLaMA3.
  • Figure 5: The ablation study of WorldCup ($k=2$) on LLaMA3.
  • ...and 7 more figures

Theorems & Definitions (9)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Definition 3.4
  • Proposition 3.5
  • Theorem 7.1
  • Lemma 7.2
  • Theorem 7.3
  • Definition 9.1