GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesis

TL;DR

Groot presents a proactive, diffusion-model-based approach to watermarking audio, embedding and later extracting watermarks directly within the diffusion generation pipeline by training a lightweight encoder and a robust decoder around a fixed DM. The method achieves high fidelity and scalable capacity (up to $5000$ bps) while demonstrating strong robustness against both individual and compound post-processing attacks, with extraction accuracies around $95\%$ on average under challenging conditions. Key contributions include a plug-and-play watermarking paradigm for diffusion-based audio, a joint optimization framework that preserves audio quality, and a formal watermark verification procedure via a binomial test. The results indicate Groot’s practical potential for regulating synthesized audio and tracing its source models in real-world deployment.

Abstract

Amid the burgeoning development of generative models like diffusion models, the task of differentiating synthesized audio from its natural counterpart grows more daunting. Deepfake detection offers a viable solution to combat this challenge. Yet, this defensive measure unintentionally fuels the continued refinement of generative models. Watermarking emerges as a proactive and sustainable tactic, preemptively regulating the creation and dissemination of synthesized content. Thus, this paper, as a pioneer, proposes the generative robust audio watermarking method (Groot), presenting a paradigm for proactively supervising the synthesized audio and its source diffusion models. In this paradigm, the processes of watermark generation and audio synthesis occur simultaneously, facilitated by parameter-fixed diffusion models equipped with a dedicated encoder. The watermark embedded within the audio can subsequently be retrieved by a lightweight decoder. The experimental results highlight Groot's outstanding performance, particularly in terms of robustness, surpassing that of the leading state-of-the-art methods. Beyond its impressive resilience against individual post-processing attacks, Groot exhibits exceptional robustness when facing compound attacks, maintaining an average watermark extraction accuracy of around 95%.

Figures (7)

• Figure 1: The diagram illustrates the process of supervising generated content through generative watermarking. The synthetic content generated via GMs by Alice will be subject to regulation, while Bob's may pose a high risk to society.
• Figure 2: The pipeline of Groot. a) Training Process, where the watermark $\mathbf{w}$ is encoded into a latent variable $\mathbf{\sigma}$ using the encoder $\mathbf{E}(\cdot)$. A Gaussian latent variable $\mathbf s_T$ is sampled from a standard distribution. Watermarked audio is then generated from the final latent variable by adding $\mathbf s_T$ and $\mathbf{\sigma}$ via diffusion models, employing the mel-spectrogram as a condition. The extracting stage employs the watermark decoder $\mathbf{D}(\cdot)$ to recover the watermark $\hat{\mathbf{w}}$ from the watermarked audio. b) Inference Process, where the watermark from the encoder is directly used by the diffusion model to synthesize the watermarked audio, eliminating the need for additional Gaussian latent variables.
• Figure 3: Architecture of the Encoder and Decoder.
• Figure 4: A test result for binomial distribution under hypothesis $H_0$ and $H_1$ for $\xi$ =0.5442 and $\xi$ = 0.9969. Given the total number of samples is 768, an FPR$\leq$0.0037 and the threshold will be $T_S \in [0.61, 0.99]$. With the threshold, FNR$\leq$0.012.
• Figure 5: The Analysis of Capacity Under Various Datasets.