Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self Voice Conversion as an Attack against Neural Audio Watermarking

Yigitcan Özer, Wanying Ge, Zhe Zhang, Xin Wang, Junichi Yamagishi

TL;DR

This paper addresses the vulnerability of neural audio watermarking to content-preserving attacks that operate in latent representation spaces. It introduces self voice conversion (VC) as a universal attack that preserves linguistic content and speaker identity while perturbing latent factors to suppress embedded watermarks, and evaluates two VC approaches (kNN-VC and RVC) against five watermarking baselines. The experiments show that self VC drives watermark detection to near-random performance, often outperforming conventional vocoder attacks while preserving intelligibility and perceptual quality. The findings reveal a fundamental mismatch between current watermarking designs and modern representation-learning pipelines, underscoring the need for watermarking methods resilient to latent-space manipulations.

Abstract

Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and digital signal processing-based watermarking methods have improved imperceptibility and embedding capacity, robustness is still primarily assessed against conventional distortions such as compression, additive noise, and resampling. However, the rise of deep learning-based attacks introduces novel and significant threats to watermark security. In this work, we investigate self voice conversion as a universal, content-preserving attack against audio watermarking systems. Self voice conversion remaps a speaker's voice to the same identity while altering acoustic characteristics through a voice conversion model. We demonstrate that this attack severely degrades the reliability of state-of-the-art watermarking approaches and highlight its implications for the security of modern audio watermarking techniques.

Self Voice Conversion as an Attack against Neural Audio Watermarking

TL;DR

This paper addresses the vulnerability of neural audio watermarking to content-preserving attacks that operate in latent representation spaces. It introduces self voice conversion (VC) as a universal attack that preserves linguistic content and speaker identity while perturbing latent factors to suppress embedded watermarks, and evaluates two VC approaches (kNN-VC and RVC) against five watermarking baselines. The experiments show that self VC drives watermark detection to near-random performance, often outperforming conventional vocoder attacks while preserving intelligibility and perceptual quality. The findings reveal a fundamental mismatch between current watermarking designs and modern representation-learning pipelines, underscoring the need for watermarking methods resilient to latent-space manipulations.

Abstract

Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and digital signal processing-based watermarking methods have improved imperceptibility and embedding capacity, robustness is still primarily assessed against conventional distortions such as compression, additive noise, and resampling. However, the rise of deep learning-based attacks introduces novel and significant threats to watermark security. In this work, we investigate self voice conversion as a universal, content-preserving attack against audio watermarking systems. Self voice conversion remaps a speaker's voice to the same identity while altering acoustic characteristics through a voice conversion model. We demonstrate that this attack severely degrades the reliability of state-of-the-art watermarking approaches and highlight its implications for the security of modern audio watermarking techniques.
Paper Structure (15 sections, 2 figures, 2 tables)

This paper contains 15 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Audio Watermarking Approaches
  4. From Channel Distortions to Malicious Neural Attacks
  5. Transmission Channel Distortions
  6. Attacker-Side Transformations
  7. Proposed Attack Method: Self Voice Conversion (VC)
  8. Background and Attack Rationale
  9. Threat Model and Attacker's Motivations
  10. kNN-VC: VC with Just Nearest Neighbors BaasNK23_VoiceConversionNN_Interspeech
  11. Retrieval Based Voice Conversion (RVC)
  12. Evaluation
  13. Discussion
  14. Conclusion
  15. Acknowledgements

Figures (2)

  • Figure 1: Illustration of the end-to-end watermarking pipeline under the proposed self voice conversion (VC) attack. The embedder inserts a multi-bit watermark into bona fide speech, which may first undergo transmission channel distortions before reaching the attacker. The attacker applies self VC to regenerate the signal using the same speaker identity, thereby perturbing or suppressing the embedded watermark. Additional transmission channel distortions may also occur after the attacker’s operation, compounding the degradation before the regenerated audio reaches the watermark (WM) detector.
  • Figure 2: Overview of the proposed self VC pipeline. Input watermarked (WM) speech is decomposed into speaker identity, content features, and, optionally, pitch contour (extracted via a pitch tracker, shown with dotted lines). These representations are fused by the decoder to generate a mel spectrogram, which is then synthesized into waveform by a vocoder to reconstruct the WM speech signal.