SEAL: Subspace-Anchored Watermarks for LLM Ownership

TL;DR

SEAL addresses the challenging problem of protecting ownership of large language models by embedding multi-bit watermarks directly into the models' latent representations using model editing. It departs from traditional backdoor or weight-based schemes by creating a latent, subspace watermark anchored to factual knowledge, enabling both white-box and black-box verification, with Bayesian reanchoring to combat drift from post-deployment updates. Empirical results across six LLMs show perfect lineage identification (AUC = 1.00), robust multi-bit extraction up to 1024 bits with minimal bit-error rates in both verification modes, and preserved model utility under various attacks. The approach is scalable, memory-efficient, and practical for real-world IP protection in API-based deployments, offering a strong, resilient foundation for LLM ownership verification and enforcement.

Abstract

Large language models (LLMs) have achieved remarkable success across a wide range of natural language processing tasks, demonstrating human-level performance in text generation, reasoning, and question answering. However, training such models requires substantial computational resources, large curated datasets, and sophisticated alignment procedures. As a result, they constitute highly valuable intellectual property (IP) assets that warrant robust protection mechanisms. Existing IP protection approaches suffer from critical limitations. Model fingerprinting techniques can identify model architectures but fail to establish ownership of specific model instances. In contrast, traditional backdoor-based watermarking methods embed behavioral anomalies that can be easily removed through common post-processing operations such as fine-tuning or knowledge distillation. We propose SEAL, a subspace-anchored watermarking framework that embeds multi-bit signatures directly into the model's latent representational space, supporting both white-box and black-box verification scenarios. Our approach leverages model editing techniques to align the hidden representations of selected anchor samples with predefined orthogonal bit vectors. This alignment embeds the watermark while preserving the model's original factual predictions, rendering the watermark functionally harmless and stealthy. We conduct comprehensive experiments on multiple benchmark datasets and six prominent LLMs, comparing SEAL with 11 existing fingerprinting and watermarking methods to demonstrate its superior effectiveness, fidelity, efficiency, and robustness. Furthermore, we evaluate SEAL under potential knowledgeable attacks and show that it maintains strong verification performance even when adversaries possess knowledge of the watermarking mechanism and the embedded signatures.

Figures (5)

• Figure 1: Method overview of SEAL in comparsion with existing backdoor-based behvioral watermarking methods. Following this, we enable encoding multiple secret bits (e.g., 64, 256)” as watermarking of a LLM instance.
• Figure 2: The overall workflow of SEAL.
• Figure 3: Scalability of SEAL in extracting multi-bit watermarks with increasing bit length under SFT attacks. We report the BER and the number of incorrectly extracted bits for each bit length.
• Figure 4: Efficiency study of watermark insertion and verification for different bit lengths in runtime.
• Figure 5: GPU memory usage (GB) during watermark insertion and verification.