Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HATS: High-Accuracy Triple-Set Watermarking for Large Language Models

Zhiqing Hu, Chenxu Zhao, Jiazhong Lu, Xiaolei Liu

TL;DR

HATS advances LLM watermarking by introducing a three-color (green/yellow/red) token partitioning that strengthens detection robustness without severely compromising text quality. The method extends KGW with a Poisson–binomial-based detection framework and Fisher-combined evidence from Green enrichment and Red depletion, enabling more reliable provenance verification. Empirical results on Llama 2 7B show superior true-positive rates and lower false-positive rates compared to prior methods, with acceptable perplexity performance. This approach enhances practical traceability of generated content in real-world, diverse linguistic contexts.

Abstract

Misuse of LLM-generated text can be curbed by watermarking techniques that embed implicit signals into the output. We propose a watermark that partitions the vocabulary at each decoding step into three sets (Green/Yellow/Red) with fixed ratios and restricts sampling to the Green and Yellow sets. At detection time, we replay the same partitions, compute Green-enrichment and Red-depletion statistics, convert them to one-sided z-scores, and aggregate their p-values via Fisher's method to decide whether a passage is watermarked. We implement generation, detection, and testing on Llama 2 7B, and evaluate true-positive rate, false-positive rate, and text quality. Results show that the triple-partition scheme achieves high detection accuracy at fixed FPR while preserving readability.

HATS: High-Accuracy Triple-Set Watermarking for Large Language Models

TL;DR

HATS advances LLM watermarking by introducing a three-color (green/yellow/red) token partitioning that strengthens detection robustness without severely compromising text quality. The method extends KGW with a Poisson–binomial-based detection framework and Fisher-combined evidence from Green enrichment and Red depletion, enabling more reliable provenance verification. Empirical results on Llama 2 7B show superior true-positive rates and lower false-positive rates compared to prior methods, with acceptable perplexity performance. This approach enhances practical traceability of generated content in real-world, diverse linguistic contexts.

Abstract

Misuse of LLM-generated text can be curbed by watermarking techniques that embed implicit signals into the output. We propose a watermark that partitions the vocabulary at each decoding step into three sets (Green/Yellow/Red) with fixed ratios and restricts sampling to the Green and Yellow sets. At detection time, we replay the same partitions, compute Green-enrichment and Red-depletion statistics, convert them to one-sided z-scores, and aggregate their p-values via Fisher's method to decide whether a passage is watermarked. We implement generation, detection, and testing on Llama 2 7B, and evaluate true-positive rate, false-positive rate, and text quality. Results show that the triple-partition scheme achieves high detection accuracy at fixed FPR while preserving readability.
Paper Structure (11 sections, 5 equations, 2 figures, 2 tables, 2 algorithms)

This paper contains 11 sections, 5 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Review of the KGW Method
  5. Generation Stage
  6. Detection Stage
  7. Experiments
  8. Experimental Setup
  9. True Positive Rate
  10. False Positive Rate
  11. Text Quality

Figures (2)

  • Figure 1: Overall framework of the proposed method.
  • Figure 2: Perplexity-based best-count (number of texts) for KGW, KGW2, OpenAI, Marylandz, and HATS. Lower perplexity indicates better alignment with the model’s language distribution.