HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models

TL;DR

HMark addresses unauthorized data use in diffusion-model training by embedding multi-bit, semantic-latent watermarks directly in the h-space of reverse diffusion. An autoencoder learns latent residuals that encode secret bits and injects them at the final timestep, yielding a radioactive watermark that persists through finetuning while maintaining high perceptual fidelity. The approach achieves near-perfect watermark detection (≈98–99%), high bit-recovery accuracy (≈87–98%), and robust performance under common distortions, with optional BCH error-correction to bolster bit recovery for larger bit-lengths. These results demonstrate a practical, verifiable method for dataset ownership protection and IP tracing in generative AI pipelines. The work also discusses limitations and situates itself within the broader landscape of watermarking, cloaking, and data-poisoning defenses.

Abstract

Modern generative diffusion models rely on vast training datasets, often including images with uncertain ownership or usage rights. Radioactive watermarks -- marks that transfer to a model's outputs -- can help detect when such unauthorized data has been used for training. Moreover, aside from being radioactive, an effective watermark for protecting images from unauthorized training also needs to meet other existing requirements, such as imperceptibility, robustness, and multi-bit capacity. To overcome these challenges, we propose HMARK, a novel multi-bit watermarking scheme, which encodes ownership information as secret bits in the semantic-latent space (h-space) for image diffusion models. By leveraging the interpretability and semantic significance of h-space, ensuring that watermark signals correspond to meaningful semantic attributes, the watermarks embedded by HMARK exhibit radioactivity, robustness to distortions, and minimal impact on perceptual quality. Experimental results demonstrate that HMARK achieves 98.57% watermark detection accuracy, 95.07% bit-level recovery accuracy, 100% recall rate, and 1.0 AUC on images produced by the downstream adversarial model finetuned with LoRA on watermarked data across various types of distortions.

Figures (9)

• Figure 1: Visual comparison between original and watermarked images. Each pair shows the original (left) and watermarked (right) images.
• Figure 2: HMark's pipeline: (1) HMark trains a watermarking encoder--decoder on a set of images X and random secrets; (2) Data owner utilizes HMark to embed watermarks on their proprietary data $\mathcal{D}_{\text{orig}}$ for IP protection before publishing; (3) The adversary finetunes $\mathcal{M}_{\text{adv}}$ on data owner's watermark-protected $\mathcal{D}_{\text{wm}}$ scraped online; (4) Data owner utilizes HMark to detect potential IP infringement in $\mathcal{D}_{\text{adv}}$.
• Figure 3: ROC curves of Recipe versus HMark at Stage 2 and Stage 4 when $\mathcal{M}_{\text{adv}}$ is LoRA finetuned.
• Figure 4: Impact of $|\mathcal{D}_{\text{wm}}|$ on $\text{Acc}_{\text{bit}}$ and the evolution of HMark's performance as the number of finetuning steps increases when $\mathcal{M}_{\text{adv}}$ is LoRA finetuned.
• Figure 6: Comparison of generations from $\mathcal{M}_{\text{adv}}$ finetuned with LoRA ($\texttt{rank}=32$) across finetuning steps when trained on (top) $\mathcal{D}_{\text{orig}}$, (second) $\mathcal{D}_{\text{wm}}$ with 8-bit secret, (third) $\mathcal{D}_{\text{wm}}$ with 16-bit secret, and (bottom) $\mathcal{D}_{\text{wm}}$ with 32-bit secret.