Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embedding Watermarks in Diffusion Process for Model Intellectual Property Protection

Jijia Yang, Sen Peng, Xiaohua Jia

TL;DR

This work introduces a novel watermarking framework by embedding the watermark into the whole diffusion process, and theoretically ensure that the final output samples contain no additional information, and utilizes statistical algorithms to verify the watermark from internally generated model samples without necessitating triggers as conditions.

Abstract

In practical application, the widespread deployment of diffusion models often necessitates substantial investment in training. As diffusion models find increasingly diverse applications, concerns about potential misuse highlight the imperative for robust intellectual property protection. Current protection strategies either employ backdoor-based methods, integrating a watermark task as a simpler training objective with the main model task, or embedding watermarks directly into the final output samples. However, the former approach is fragile compared to existing backdoor defense techniques, while the latter fundamentally alters the expected output. In this work, we introduce a novel watermarking framework by embedding the watermark into the whole diffusion process, and theoretically ensure that our final output samples contain no additional information. Furthermore, we utilize statistical algorithms to verify the watermark from internally generated model samples without necessitating triggers as conditions. Detailed theoretical analysis and experimental validation demonstrate the effectiveness of our proposed method.

Embedding Watermarks in Diffusion Process for Model Intellectual Property Protection

TL;DR

This work introduces a novel watermarking framework by embedding the watermark into the whole diffusion process, and theoretically ensure that the final output samples contain no additional information, and utilizes statistical algorithms to verify the watermark from internally generated model samples without necessitating triggers as conditions.

Abstract

In practical application, the widespread deployment of diffusion models often necessitates substantial investment in training. As diffusion models find increasingly diverse applications, concerns about potential misuse highlight the imperative for robust intellectual property protection. Current protection strategies either employ backdoor-based methods, integrating a watermark task as a simpler training objective with the main model task, or embedding watermarks directly into the final output samples. However, the former approach is fragile compared to existing backdoor defense techniques, while the latter fundamentally alters the expected output. In this work, we introduce a novel watermarking framework by embedding the watermark into the whole diffusion process, and theoretically ensure that our final output samples contain no additional information. Furthermore, we utilize statistical algorithms to verify the watermark from internally generated model samples without necessitating triggers as conditions. Detailed theoretical analysis and experimental validation demonstrate the effectiveness of our proposed method.

Paper Structure

This paper contains 30 sections, 2 theorems, 20 equations, 7 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Diffusion Probabilistic Models
  4. Watermarking Discriminative Models
  5. Watermarking Generative Models
  6. Preliminary and Problem Definition
  7. Diffusion Process
  8. Threat Model
  9. Problem Formulation
  10. Embedding Watermarks in Intermediate Diffusion Process
  11. Theoretical Analysis
  12. Watermark Embedding
  13. Watermark Verification
  14. Experimental Setup
  15. Datasets
  16. ...and 15 more sections

Key Result

Theorem 1

For samples $x_0 \sim q(x)$, $\{x_t\}_{t\in[1,t_A]}$ represents the sequence of the noised samples processed by DM, a sequence $\{b_t\}$, $\gamma \in (0,1]$ and for $t\in [0,t_A]$: the corresponding conditional probability $q({x_{t-1}}^\prime | {x_t}^\prime,{x_0}^\prime)$ is given by eqa:q_prime, where ${\sigma^\prime}^2$, $\mu^\prime$ and $\epsilon_t^{\prime\prime}$ are given by and eqa:sigma_pr

Figures (7)

  • Figure 1: Modified training process when watermark step $t_A = T/2$.
  • Figure 2: Different watermark settings (shape & position).
  • Figure 3: Watermarked sampling process with average results of 100 samples among different timesteps t performed on different datasets.
  • Figure 4: Comparison between samples generated by baseline models and watermarked models.
  • Figure 5: Watermark verification.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • proof