Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TATTOOED: A Robust Deep Neural Network Watermarking Scheme based on Spread-Spectrum Channel Coding

Giulio Pagnotta, Dorjan Hitaj, Briland Hitaj, Fernando Perez-Cruz, Luigi V. Mancini

TL;DR

TATTOOED introduces a CDMA spread-spectrum watermarking scheme for deep neural networks that embeds a multi-bit watermark directly into model weights post-training. It combines an LDPC encoder with CDMA spreading to encode the watermark into a subset of parameters, producing $W_{wtm} = W + \gamma \mathbf{C}\mathbf{m}$, and retrieves it via CDMA/CDMA decoding followed by LDPC decoding. Across image, text, and audio domains and multiple architectures, it achieves near-zero degradation in task performance and remains BER=0 under strong attacks like RTAL, FTAL, and REFIT, while resisting pruning, shuffling, and overwriting due to the spread-spectrum encoding. The results indicate that TATTOOED offers a robust, general, and secure approach for DNN IP protection with practical post-training deployment and broad applicability to modern models including large-scale ones.

Abstract

Watermarking of deep neural networks (DNNs) has gained significant traction in recent years, with numerous (watermarking) strategies being proposed as mechanisms that can help verify the ownership of a DNN in scenarios where these models are obtained without the permission of the owner. However, a growing body of work has demonstrated that existing watermarking mechanisms are highly susceptible to removal techniques, such as fine-tuning, parameter pruning, or shuffling. In this paper, we build upon extensive prior work on covert (military) communication and propose TATTOOED, a novel DNN watermarking technique that is robust to existing threats. We demonstrate that using TATTOOED as their watermarking mechanisms, the DNN owner can successfully obtain the watermark and verify model ownership even in scenarios where 99% of model parameters are altered. Furthermore, we show that TATTOOED is easy to employ in training pipelines, and has negligible impact on model performance.

TATTOOED: A Robust Deep Neural Network Watermarking Scheme based on Spread-Spectrum Channel Coding

TL;DR

TATTOOED introduces a CDMA spread-spectrum watermarking scheme for deep neural networks that embeds a multi-bit watermark directly into model weights post-training. It combines an LDPC encoder with CDMA spreading to encode the watermark into a subset of parameters, producing , and retrieves it via CDMA/CDMA decoding followed by LDPC decoding. Across image, text, and audio domains and multiple architectures, it achieves near-zero degradation in task performance and remains BER=0 under strong attacks like RTAL, FTAL, and REFIT, while resisting pruning, shuffling, and overwriting due to the spread-spectrum encoding. The results indicate that TATTOOED offers a robust, general, and secure approach for DNN IP protection with practical post-training deployment and broad applicability to modern models including large-scale ones.

Abstract

Watermarking of deep neural networks (DNNs) has gained significant traction in recent years, with numerous (watermarking) strategies being proposed as mechanisms that can help verify the ownership of a DNN in scenarios where these models are obtained without the permission of the owner. However, a growing body of work has demonstrated that existing watermarking mechanisms are highly susceptible to removal techniques, such as fine-tuning, parameter pruning, or shuffling. In this paper, we build upon extensive prior work on covert (military) communication and propose TATTOOED, a novel DNN watermarking technique that is robust to existing threats. We demonstrate that using TATTOOED as their watermarking mechanisms, the DNN owner can successfully obtain the watermark and verify model ownership even in scenarios where 99% of model parameters are altered. Furthermore, we show that TATTOOED is easy to employ in training pipelines, and has negligible impact on model performance.
Paper Structure (24 sections, 5 equations, 3 figures, 5 tables, 2 algorithms)

This paper contains 24 sections, 5 equations, 3 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. DNN Model Watermarking
  4. Spread-Spectrum Channel Coding
  5. Threat Model
  6. TATTOOED
  7. Experimental Setup
  8. Datasets
  9. DNN Architectures
  10. Watermarks
  11. parameters
  12. Evaluation
  13. Fidelity
  14. Robustness
  15. Security
  16. ...and 9 more sections

Figures (3)

  • Figure 1: A high-level overview of TATTOOED watermarking scheme. To watermark a DNN (\ref{['fig:scheme_mark']}), TATTOOED uses a secret key k, out of which two separate keys (ld[k] and sp[k]) are generated. ld[k] is used in the LDPC encoder (and decoder on extraction procedure) and sp[k] is used to generate the spreading codes (both in CDMA encoder and decoder). The LDPC encoder takes the watermark k, encodes it, and then passes it to the CDMA encoder, which spreads the LDPC-encoded watermark in a wider bandwidth and then adds it the selected parameters of model $W$ to obtain the watermarked model $W_{wtm}$. To extract the watermark (\ref{['fig:scheme_verify']}), a watermarked model using TATTOOED is given as input to the Extract module, passing first to the CDMA decoder, and then to the LDPC decoder to recover the hidden watermark m'.
  • Figure 2: The effect of RTAL on watermark using the same amount of additional data to retrain the model as that used for initial training. In each case, the watermark BER is equal to 0.
  • Figure 3: The effect of model fine-tuning on watermark using a different dataset than the one used for the training. In each case, after the fine-tuning procedure, the watermark BER was equal to 0.