Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RobWE: Robust Watermark Embedding for Personalized Federated Learning Model Ownership Protection

Yang Xu, Yunlin Tan, Cheng Zhang, Kai Chi, Peng Sun, Wenyuan Yang, Ju Ren, Hongbo Jiang, Yaoxue Zhang

TL;DR

RobWE tackles ownership protection for personalized models in personalized federated learning by decoupling watermark embedding into a private head layer and a shared representation layer, and by introducing watermark slice embedding to avoid conflicts during aggregation. A server-side tampered watermark detector validates watermarks before aggregation, deterring malicious tampering and collusion attempts. Empirical results on standard datasets show RobWE outperforms prior FL watermarking approaches in fidelity, reliability, and robustness, including resilience to pruning, fine-tuning, and adaptive tampering. This work enables practical ownership verification in PFL and points to future directions in tamper-resistant embedding and leakage tracing.

Abstract

Embedding watermarks into models has been widely used to protect model ownership in federated learning (FL). However, existing methods are inadequate for protecting the ownership of personalized models acquired by clients in personalized FL (PFL). This is due to the aggregation of the global model in PFL, resulting in conflicts over clients' private watermarks. Moreover, malicious clients may tamper with embedded watermarks to facilitate model leakage and evade accountability. This paper presents a robust watermark embedding scheme, named RobWE, to protect the ownership of personalized models in PFL. We first decouple the watermark embedding of personalized models into two parts: head layer embedding and representation layer embedding. The head layer belongs to clients' private part without participating in model aggregation, while the representation layer is the shared part for aggregation. For representation layer embedding, we employ a watermark slice embedding operation, which avoids watermark embedding conflicts. Furthermore, we design a malicious watermark detection scheme enabling the server to verify the correctness of watermarks before aggregating local models. We conduct an exhaustive experimental evaluation of RobWE. The results demonstrate that RobWE significantly outperforms the state-of-the-art watermark embedding schemes in FL in terms of fidelity, reliability, and robustness.

RobWE: Robust Watermark Embedding for Personalized Federated Learning Model Ownership Protection

TL;DR

RobWE tackles ownership protection for personalized models in personalized federated learning by decoupling watermark embedding into a private head layer and a shared representation layer, and by introducing watermark slice embedding to avoid conflicts during aggregation. A server-side tampered watermark detector validates watermarks before aggregation, deterring malicious tampering and collusion attempts. Empirical results on standard datasets show RobWE outperforms prior FL watermarking approaches in fidelity, reliability, and robustness, including resilience to pruning, fine-tuning, and adaptive tampering. This work enables practical ownership verification in PFL and points to future directions in tamper-resistant embedding and leakage tracing.

Abstract

Embedding watermarks into models has been widely used to protect model ownership in federated learning (FL). However, existing methods are inadequate for protecting the ownership of personalized models acquired by clients in personalized FL (PFL). This is due to the aggregation of the global model in PFL, resulting in conflicts over clients' private watermarks. Moreover, malicious clients may tamper with embedded watermarks to facilitate model leakage and evade accountability. This paper presents a robust watermark embedding scheme, named RobWE, to protect the ownership of personalized models in PFL. We first decouple the watermark embedding of personalized models into two parts: head layer embedding and representation layer embedding. The head layer belongs to clients' private part without participating in model aggregation, while the representation layer is the shared part for aggregation. For representation layer embedding, we employ a watermark slice embedding operation, which avoids watermark embedding conflicts. Furthermore, we design a malicious watermark detection scheme enabling the server to verify the correctness of watermarks before aggregating local models. We conduct an exhaustive experimental evaluation of RobWE. The results demonstrate that RobWE significantly outperforms the state-of-the-art watermark embedding schemes in FL in terms of fidelity, reliability, and robustness.
Paper Structure (20 sections, 1 equation, 5 figures, 3 tables, 1 algorithm)

This paper contains 20 sections, 1 equation, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Watermarking in Centralized Learning
  4. Watermarking in Federated Learning
  5. Problem Statement
  6. Tampering Attack
  7. Adaptive Tampering Attack
  8. Problem Definition
  9. Proposed Scheme
  10. Overview of RobWE
  11. System Setup
  12. Watermark Decoupled Embedding
  13. Tampered Watermark Detection
  14. Experiments
  15. Experimental Setup
  16. ...and 5 more sections

Figures (5)

  • Figure 1: In the Non-IID settings (i.e., Dir(0.5) and K(4)), the watermark detection rates of malicious clients (solid line) and honest clients (dashed line) in FedIPR and RobWE (without defense).
  • Figure 2: An illustration of RobWE.
  • Figure 3: The watermark detection rate for other client watermarks on each client' model (non-diagonal region) and the detection rate for own private watermarks (diagonal region).
  • Figure 4: In the Non-IID setting, the model accuracy (dashed line) and watermark detection rates (solid line) of models with different embedded bits watermark after attacks.
  • Figure 5: The Quantile-Quantile plot for assessing whether the watermark detection rate between normal clients and attackers obeys a normal distribution.