CBW: Towards Dataset Ownership Verification for Speaker Verification via Clustering-based Backdoor Watermarking

TL;DR

The paper addresses copyright protection for public speaker verification datasets by proposing clustering-based backdoor watermarking (CBW) and a hypothesis-test–based ownership verification framework suitable for black-box model evaluation. CBW watermarks data by clustering speakers in feature space and implanting cluster-specific triggers, enabling distinctive backdoor behavior without exposing enrolled speakers, and it is paired with both similarity-available and decision-only verification modes. The authors provide theoretical analyses showing verification viability with practical watermarking rates and conduct extensive experiments across multiple models and datasets, demonstrating high watermark success rates and low impact on normal performance, along with robustness to adaptive attacks and transferability across models. The work offers a practical, scalable approach to trustworthy data sharing and licensing in biometric verification, with potential extensions to other verification tasks. Overall, CBW advances dataset copyright protection by marrying clustered trigger design with rigorous statistical verification in a black-box setting, supported by empirical and theoretical guarantees.

Abstract

With the increasing adoption of deep learning in speaker verification, large-scale speech datasets have become valuable intellectual property. To audit and prevent the unauthorized usage of these valuable released datasets, especially in commercial or open-source scenarios, we propose a novel dataset ownership verification method. Our approach introduces a clustering-based backdoor watermark (CBW), enabling dataset owners to determine whether a suspicious third-party model has been trained on a protected dataset under a black-box setting. The CBW method consists of two key stages: dataset watermarking and ownership verification. During watermarking, we implant multiple trigger patterns in the dataset to make similar samples (measured by their feature similarities) close to the same trigger while dissimilar samples are near different triggers. This ensures that any model trained on the watermarked dataset exhibits specific misclassification behaviors when exposed to trigger-embedded inputs. To verify dataset ownership, we design a hypothesis-test-based framework that statistically evaluates whether a suspicious model exhibits the expected backdoor behavior. We conduct extensive experiments on benchmark datasets, verifying the effectiveness and robustness of our method against potential adaptive attacks. The code for reproducing main experiments is available at https://github.com/Radiant0726/CBW

Figures (8)

• Figure 1: The comparison between speaker classification and speaker verification. In general, speaker classification intends to identify which pre-defined speaker a test audio belongs to, while speaker verification determines whether the audio is from enrolled speakers. The gray background indicates that the potential speaker of the test audio has appeared in the training dataset of classification tasks. In contrast, the potential enrolled test speakers in verification tasks are generally not involved in the training dataset.
• Figure 2: The main pipeline of dataset ownership verification for speaker verification via our clustering-based backdoor watermark (CBW). In general, our CBW consists of three main steps: (1) feature extraction, (2) speaker clustering, and (3) trigger implanting. In the first step, we obtain the feature representation of each sample based on a (pre-trained) surrogate benign model. After that, we cluster all the speakers into $K$ clusters based on the similarity of their average feature representations and implant respective trigger patterns in each cluster. All models trained on the CBW-watermarked dataset will behave normally on benign samples, while the sequence of all pre-defined unenrolled triggers will likely pass the verification. As such, we can design a hypothesis test-guided dataset ownership verification based on our CBW to detect whether a suspicious model was trained on the CBW-protected dataset based on model's predictions on trigger sequence under similarity-available and acceptance-only verification settings in single and multiple enrollment scenarios.
• Figure 3: The example of CBW-watermarked audios.
• Figure 4: The WSR (%) and EER (%) of our CBW $w.r.t.$ different number of clusters ($i.e.$, $K$) on the TIMIT dataset.
• Figure 5: The WSR (%) and EER (%) of our CBW $w.r.t.$ different trigger volumns on the TIMIT dataset.

Theorems & Definitions (5)

• Proposition 1: Similarity-available Verification
• Proposition 2: Decision-only Verification
• Theorem 1
• Theorem 1
• proof