EditMark: Watermarking Large Language Models based on Model Editing

TL;DR

EditMark addresses copyright protection for open-source LLMs by embedding watermarks directly into model weights through a training-free, model-editing-based pipeline. It leverages the diversity of multiple-answer questions (MAQ) to encode a watermark as permutations of target answers, and enhances robustness with an adaptive multi-round editing strategy and a noise matrix. The framework combines Generator, Encoder, Editor, and Decoder to train watermarks quickly (e.g., 32-bit under 20 seconds) with 100% extraction reliability and minimal impact on downstream task performance, while remaining stealthy. Empirical results show strong effectiveness, fidelity, stealthiness, and resilience against common attacks, demonstrating practical viability for LLM provenance and copyright enforcement.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities, but their training requires extensive data and computational resources, rendering them valuable digital assets. Therefore, it is essential to watermark LLMs to protect their copyright and trace unauthorized use or resale. Existing methods for watermarking LLMs primarily rely on training LLMs with a watermarked dataset, which entails burdensome training costs and negatively impacts the LLM's performance. In addition, their watermarked texts are not logical or natural, thereby reducing the stealthiness of the watermark. To address these issues, we propose EditMark, the first watermarking method that leverages model editing to embed a training-free, stealthy, and performance-lossless watermark for LLMs. We observe that some questions have multiple correct answers. Therefore, we assign each answer a unique watermark and update the weights of LLMs to generate corresponding questions and answers through the model editing technique. In addition, we refine the model editing technique to align with the requirements of watermark embedding. Specifically, we introduce an adaptive multi-round stable editing strategy, coupled with the injection of a noise matrix, to improve both the effectiveness and robustness of the watermark embedding. Extensive experiments indicate that EditMark can embed 32-bit watermarks into LLMs within 20 seconds (Fine-tuning: 6875 seconds) with a watermark extraction success rate of 100%, which demonstrates its effectiveness and efficiency. External experiments further demonstrate that EditMark has fidelity, stealthiness, and a certain degree of robustness against common attacks.

Figures (11)

• Figure 1: The watermarked texts of different watermarking methods.
• Figure 2: The pipeline of EditMark. The Generator uses a seed to generate a random integer sequence $\mathcal{A} = (a_1, a_2, \dots, a_u)$, and then generates the MA question based on the MA question template. The Encoder derives the target answers for the MA questions using the encoding function $f$ and the watermark $\mathcal{W}$. The Editor edits the LLM based on the QA pair to make it generate the target answer to the corresponding MA question. The Decoder obtains the answer to the MA question and extracts the watermark from the resulting QA pair using the decoding function $g$.
• Figure 3: Distribution of the differences in $\bm{K}_1$ for the LLMs before and after random noise and pruning attacks. The above is a random noise attack, and the below is a model pruning attack.
• Figure 4: ESR of original watermarked LLMs and the attacked LLMs under model fine-tuning attacks.
• Figure 5: The ESR of original watermarked LLMs and the attacked LLM by quantification.