Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesis

Lauri Juvela, Xin Wang

TL;DR

The paper tackles robust detection of synthetic speech in real-world distributions by integrating active watermarking with codec-aware training. It introduces a codec-augmented collaborative watermarking framework that uses a straight-through estimator to enable gradient flow through non-differentiable codecs, including both traditional codecs (MP3, Opus) and a neural DAC model. The results show improvements in robustness for both passive observers and active collaborators, with the neural codec augmentation transferring well to traditional codecs and listening tests indicating minimal perceptual degradation at high bitrates or with the DAC at 8 kbps. These findings support practical deployment of watermark-based content labeling in real-world speech synthesis systems and motivate broader codec coverage and robustness benchmarking.

Abstract

Automatic detection of synthetic speech is becoming increasingly important as current synthesis methods are both near indistinguishable from human speech and widely accessible to the public. Audio watermarking and other active disclosure methods of are attracting research activity, as they can complement traditional deepfake defenses based on passive detection. In both active and passive detection, robustness is of major interest. Traditional audio watermarks are particularly susceptible to removal attacks by audio codec application. Most generated speech and audio content released into the wild passes through an audio codec purely as a distribution method. We recently proposed collaborative watermarking as method for making generated speech more easily detectable over a noisy but differentiable transmission channel. This paper extends the channel augmentation to work with non-differentiable traditional audio codecs and neural audio codecs and evaluates transferability and effect of codec bitrate over various configurations. The results show that collaborative watermarking can be reliably augmented by black-box audio codecs using a waveform-domain straight-through-estimator for gradient approximation. Furthermore, that results show that channel augmentation with a neural audio codec transfers well to traditional codecs. Listening tests demonstrate collaborative watermarking incurs negligible perceptual degradation with high bitrate codecs or DAC at 8kbps.

Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesis

TL;DR

The paper tackles robust detection of synthetic speech in real-world distributions by integrating active watermarking with codec-aware training. It introduces a codec-augmented collaborative watermarking framework that uses a straight-through estimator to enable gradient flow through non-differentiable codecs, including both traditional codecs (MP3, Opus) and a neural DAC model. The results show improvements in robustness for both passive observers and active collaborators, with the neural codec augmentation transferring well to traditional codecs and listening tests indicating minimal perceptual degradation at high bitrates or with the DAC at 8 kbps. These findings support practical deployment of watermark-based content labeling in real-world speech synthesis systems and motivate broader codec coverage and robustness benchmarking.

Abstract

Automatic detection of synthetic speech is becoming increasingly important as current synthesis methods are both near indistinguishable from human speech and widely accessible to the public. Audio watermarking and other active disclosure methods of are attracting research activity, as they can complement traditional deepfake defenses based on passive detection. In both active and passive detection, robustness is of major interest. Traditional audio watermarks are particularly susceptible to removal attacks by audio codec application. Most generated speech and audio content released into the wild passes through an audio codec purely as a distribution method. We recently proposed collaborative watermarking as method for making generated speech more easily detectable over a noisy but differentiable transmission channel. This paper extends the channel augmentation to work with non-differentiable traditional audio codecs and neural audio codecs and evaluates transferability and effect of codec bitrate over various configurations. The results show that collaborative watermarking can be reliably augmented by black-box audio codecs using a waveform-domain straight-through-estimator for gradient approximation. Furthermore, that results show that channel augmentation with a neural audio codec transfers well to traditional codecs. Listening tests demonstrate collaborative watermarking incurs negligible perceptual degradation with high bitrate codecs or DAC at 8kbps.
Paper Structure (15 sections, 7 equations, 2 figures, 1 table)

This paper contains 15 sections, 7 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. Collaborative training
  5. Gradient approximation for codecs
  6. Experiments
  7. Dataset
  8. Models
  9. Codecs for channel augmentation
  10. Loss functions
  11. Training details
  12. Results
  13. Detection equal error rates
  14. Listening test
  15. Discussion and Conclusion

Figures (2)

  • Figure 1: Generated speech can be detected by either a passive Observer or an active Collaborator. Watermark detection is made more robust against codecs by approximately differentiable channel augmentation required for passing gradients to the generator in collaborative training.
  • Figure 2: Mean opinion scores (MOS) for speech naturalness. 'C' denotes collaborative training, with codec augmentation type in parentheses.