Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Watermark: A Selective Watermarking Framework for Large Language Models via Multi-Objective Optimization

Chenrui Wang, Junyi Shu, Billy Chiu, Yu Li, Saleh Alharbi, Min Zhang, Jing Li

TL;DR

This work tackles the trade-off between detectability and text quality in LLM watermarking by introducing Learning to Watermark (LTW), a selective framework that uses a lightweight Selector Network to decide when to watermark tokens based on sentence embeddings, token entropy, and watermarking ratio. LTW is trained with multi-objective optimization (MGDA) to jointly optimize generation quality and watermark detectability, steering toward Pareto-optimal solutions. Empirical results show that LTW achieves higher text quality (lower perplexity, preserved semantics) while maintaining strong detectability and robustness against paraphrase attacks, outperforming several baseline watermarking methods. This approach provides a practical, data-driven strategy for watermarking that can adapt to varying generation contexts while sustaining linguistic quality and traceability.

Abstract

The rapid development of LLMs has raised concerns about their potential misuse, leading to various watermarking schemes that typically offer high detectability. However, existing watermarking techniques often face trade-off between watermark detectability and generated text quality. In this paper, we introduce Learning to Watermark (LTW), a novel selective watermarking framework that leverages multi-objective optimization to effectively balance these competing goals. LTW features a lightweight network that adaptively decides when to apply the watermark by analyzing sentence embeddings, token entropy, and current watermarking ratio. Training of the network involves two specifically constructed loss functions that guide the model toward Pareto-optimal solutions, thereby harmonizing watermark detectability and text quality. By integrating LTW with two baseline watermarking methods, our experimental evaluations demonstrate that LTW significantly enhances text quality without compromising detectability. Our selective watermarking approach offers a new perspective for designing watermarks for LLMs and a way to preserve high text quality for watermarks. The code is publicly available at: https://github.com/fattyray/learning-to-watermark

Learning to Watermark: A Selective Watermarking Framework for Large Language Models via Multi-Objective Optimization

TL;DR

This work tackles the trade-off between detectability and text quality in LLM watermarking by introducing Learning to Watermark (LTW), a selective framework that uses a lightweight Selector Network to decide when to watermark tokens based on sentence embeddings, token entropy, and watermarking ratio. LTW is trained with multi-objective optimization (MGDA) to jointly optimize generation quality and watermark detectability, steering toward Pareto-optimal solutions. Empirical results show that LTW achieves higher text quality (lower perplexity, preserved semantics) while maintaining strong detectability and robustness against paraphrase attacks, outperforming several baseline watermarking methods. This approach provides a practical, data-driven strategy for watermarking that can adapt to varying generation contexts while sustaining linguistic quality and traceability.

Abstract

The rapid development of LLMs has raised concerns about their potential misuse, leading to various watermarking schemes that typically offer high detectability. However, existing watermarking techniques often face trade-off between watermark detectability and generated text quality. In this paper, we introduce Learning to Watermark (LTW), a novel selective watermarking framework that leverages multi-objective optimization to effectively balance these competing goals. LTW features a lightweight network that adaptively decides when to apply the watermark by analyzing sentence embeddings, token entropy, and current watermarking ratio. Training of the network involves two specifically constructed loss functions that guide the model toward Pareto-optimal solutions, thereby harmonizing watermark detectability and text quality. By integrating LTW with two baseline watermarking methods, our experimental evaluations demonstrate that LTW significantly enhances text quality without compromising detectability. Our selective watermarking approach offers a new perspective for designing watermarks for LLMs and a way to preserve high text quality for watermarks. The code is publicly available at: https://github.com/fattyray/learning-to-watermark
Paper Structure (44 sections, 18 equations, 11 figures, 3 tables, 3 algorithms)

This paper contains 44 sections, 18 equations, 11 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Watermarking Methods.
  4. Balancing Detection and Quality.
  5. Multi-Objective Optimization.
  6. Methodology
  7. Problem Formulation
  8. Structure Design
  9. Selector Network.
  10. Watermarking Information Collector.
  11. Adaptive Threshold.
  12. Learning to watermark: Training for Selective Watermark Insertion
  13. Preserving Quality.
  14. Maximizing Detectability.
  15. Experiments
  16. ...and 29 more sections

Figures (11)

  • Figure 1: An illustration of our selective watermarking method. (1) The upper part illustrates that at each generation time step, the watermark collector concatenates the calculated input for the trained Selector Network, whose output is compared to an adaptive threshold to decide whether to watermark the current token. (2) The lower part is about constructing the detection oriented and text quality oriented loss, and utilize the MGDA algorithm to tackle this multi-objective optimization problem and optimize towards Pareto optimal.
  • Figure 2: Comparison of the Cosine Similarity between reference text and watermarked text across various LLMs.
  • Figure 3: Distribution of perplexity among the unwatermarked text generated using OPT-6.7b and that of texts watermarked using various watermark methods.
  • Figure 4: Distribution of perplexity among the unwatermarked text generated using GPT-J-6b and that of texts watermarked using various watermark methods.
  • Figure 5: Performance comparison between our watermarking method and KGW watermark in terms of perplexity and TPR across various watermark strengths.
  • ...and 6 more figures