Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training

Houmin Sun, Zi Hu, Linxi Li, Yechen Wang, Liwei Jin, Ming Li

Abstract

Modern audio is created by mixing stems from different sources, raising the question: can we independently watermark each stem and recover all watermarks after separation? We study a separation-first, multi-stream watermarking framework-embedding distinct information into stems using unique keys but a shared structure, mixing, separating, and decoding from each output. A naive pipeline (robust watermarking + off-the-shelf separation) yields poor bit recovery, showing robustness to generic distortions does not ensure robustness to separation artifacts. To enable this, we jointly train the watermark system and the separator in an end-to-end manner, encouraging the separator to preserve watermark cues while adapting embedding to separation-specific distortions. Experiments on speech+music and vocal+accompaniment mixtures show substantial gains in post-separation recovery while maintaining perceptual quality.

Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training

Abstract

Modern audio is created by mixing stems from different sources, raising the question: can we independently watermark each stem and recover all watermarks after separation? We study a separation-first, multi-stream watermarking framework-embedding distinct information into stems using unique keys but a shared structure, mixing, separating, and decoding from each output. A naive pipeline (robust watermarking + off-the-shelf separation) yields poor bit recovery, showing robustness to generic distortions does not ensure robustness to separation artifacts. To enable this, we jointly train the watermark system and the separator in an end-to-end manner, encouraging the separator to preserve watermark cues while adapting embedding to separation-specific distortions. Experiments on speech+music and vocal+accompaniment mixtures show substantial gains in post-separation recovery while maintaining perceptual quality.
Paper Structure (18 sections, 3 equations, 1 figure, 3 tables)

This paper contains 18 sections, 3 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Problem Setup: Separation-First Multi-Stream Watermarking
  4. Watermarking Backbone
  5. Encoder.
  6. Decoder.
  7. Source Separation Network
  8. Demucs
  9. Impact of Separation on Watermarking
  10. Joint Training Pipeline
  11. Datasets and Training Setup.
  12. Loss Functions
  13. Attack Simulation.
  14. Evaluation Metrics
  15. Perceptual Quality and Separation Integrity
  16. ...and 3 more sections

Figures (1)

  • Figure 1: The joint training pipeline for the proposed separation-first audio watermarking framework. In our experiments, ${\mathbf{i}} = 1,2$