Traceable Black-box Watermarks for Federated Learning

TL;DR

This work formalizes the problem of traceable black-box watermarking in federated learning and introduces TraMark, a server-side method that creates personalized, watermarked global models by partitioning each model's parameters into a main-task region and a watermarking region. Through masked aggregation and region-constrained watermark injection, TraMark embeds distinct watermarks per client using unique watermark datasets, enabling black-box verification of ownership and traceability without degrading primary task performance. Empirical results across multiple datasets and architectures show high leakage verification rates (VR ≈ 99.6%) with only a modest average drop in main-task accuracy (≈0.54%), and robustness to pruning, fine-tuning, and quantization attacks. The approach offers practical utility for IP protection in cross-silo FL, providing a scalable, server-side mechanism to identify model leakers while preserving core FL functionality.

Abstract

Due to the distributed nature of Federated Learning (FL) systems, each local client has access to the global model, which poses a critical risk of model leakage. Existing works have explored injecting watermarks into local models to enable intellectual property protection. However, these methods either focus on non-traceable watermarks or traceable but white-box watermarks. We identify a gap in the literature regarding the formal definition of traceable black-box watermarking and the formulation of the problem of injecting such watermarks into FL systems. In this work, we first formalize the problem of injecting traceable black-box watermarks into FL. Based on the problem, we propose a novel server-side watermarking method, $\mathbf{TraMark}$, which creates a traceable watermarked model for each client, enabling verification of model leakage in black-box settings. To achieve this, $\mathbf{TraMark}$ partitions the model parameter space into two distinct regions: the main task region and the watermarking region. Subsequently, a personalized global model is constructed for each client by aggregating only the main task region while preserving the watermarking region. Each model then learns a unique watermark exclusively within the watermarking region using a distinct watermark dataset before being sent back to the local client. Extensive results across various FL systems demonstrate that $\mathbf{TraMark}$ ensures the traceability of all watermarked models while preserving their main task performance. The code is available at https://github.com/JiiahaoXU/TraMark.

Figures (7)

• Figure 1: Verification (Ver) confidence, verification leakage, verification interval, and VR on CIFAR-10 and CIFAR-100.
• Figure 2: Averaged MA and VR results of $\mathbf{TraMark}$ under pruning and fine-tuning attacks.
• Figure 3: Impact of warmup training ratio $\alpha$ on the MA and VR of $\mathbf{TraMark}$.
• Figure 4: Impact of partition ratio $k$ and watermark dataset size.
• Figure 5: The averaged KL divergence and VR over training rounds on CIFAR-10 and CIFAR-100 datasets.

Theorems & Definitions (6)

• Definition 1: Watermark Dataset
• Definition 2: Black-box Watermark
• Definition 3: Watermark Collision
• Remark 1
• Definition 4: Traceability of Watermarked Models
• Remark 2