PersonaMark: Personalized LLM watermarking for model protection and user attribution

TL;DR

PersonaMark presents a personalized text watermarking scheme for LLMs that encodes user-specific watermarks in sentence structure rather than tokens. The method leverages dependency parsing to extract sentence structures, which are hashed with per-user IDs to produce binary markers and enable scalable attribution via a dedicated hash-function database, detected through a Z-test. Experiments across four model families and multiple prompts show PersonaMark achieves perplexities close to unwatermarked baselines, preserves sentiment and readability, and delivers robust watermark detection (AUC up to 0.97 under attack). The work demonstrates that sentence-level watermarking with user-specific hashing can provide effective copyright protection and misuse attribution for customized LLMs while maintaining high text quality and model fidelity.

Abstract

The rapid advancement of customized Large Language Models (LLMs) offers considerable convenience. However, it also intensifies concerns regarding the protection of copyright/confidential information. With the extensive adoption of private LLMs, safeguarding model copyright and ensuring data privacy have become critical. Text watermarking has emerged as a viable solution for detecting AI-generated content and protecting models. However, existing methods fall short in providing individualized watermarks for each user, a critical feature for enhancing accountability and traceability. In this paper, we introduce PersonaMark, a novel personalized text watermarking scheme designed to protect LLMs' copyrights and bolster accountability. PersonaMark leverages sentence structure as a subtle carrier of watermark information and optimizes the generation process to maintain the natural output of the model. By employing a personalized hashing function, unique watermarks are embedded for each user, enabling high-quality text generation without compromising the model's performance. This approach is both time-efficient and scalable, capable of handling large numbers of users through a multi-user hashing mechanism. To the best of our knowledge, this is a pioneer study to explore personalized watermarking in LLMs. We conduct extensive evaluations across four LLMs, analyzing various metrics such as perplexity, sentiment, alignment, and readability. The results validate that PersonaMark preserves text quality, ensures unbiased watermark insertion, and offers robust watermark detection capabilities, all while maintaining the model's behavior with minimal disruption.

Figures (7)

• Figure 1: Overview of our proposed PersonaMark watermark injection framework. The italicized and underlined sentences are watermarked sentences.
• Figure 2: Overview of the personalized watermark detection process.
• Figure 3: AI-generated text detection performance on content generated by the PersonaMark. As watermarking techniques become one of the approaches for AI-generated text detection, we compare PersonaMark with five metric-based methods and a model-based method of this taskmgtbench. Results show that our method aiming for multi-user attribution still retains the ability of zero-bit watermark on AI-generated text detection , with F1 value and AUC value close to the maximum value.
• Figure 4: Watermark detection performace on four models measured by AUC curve and score. The green, orange, and blue lines denote performance of our PersonaMark method, KGW, and KGW-Persona. The titles are different LLMs' name. Our PersonaMark achieves high AUC across different models. KGW‘s AUC, when converted to KGW-Persona to fit in the peronalized setting, drops much.
• Figure 5: Watermark performance of texts genereated by Phi-3.5 model. The titles denotes the attacking strength defined by word replacement probability. The green line and orange line denote performace of our PersonaMark and KGW respectively. After the synonym attacking, the AUC score drops with the attacking strength while our PersonaMark shows better robustness than KGW.