FoldMark: Protecting Protein Generative Models with Watermarking

TL;DR

This work investigates whether it is possible to embed watermarks into protein generative models and their outputs for copyright authentication and the tracking of generated structures and proposes a two-stage method FoldMark as a generalized watermarking strategy for protein generative models.

Abstract

Protein structure is key to understanding protein function and is essential for progress in bioengineering, drug discovery, and molecular biology. Recently, with the incorporation of generative AI, the power and accuracy of computational protein structure prediction/design have been improved significantly. However, ethical concerns such as copyright protection and harmful content generation (biosecurity) pose challenges to the wide implementation of protein generative models. Here, we investigate whether it is possible to embed watermarks into protein generative models and their outputs for copyright authentication and the tracking of generated structures. As a proof of concept, we propose a two-stage method FoldMark as a generalized watermarking strategy for protein generative models. FoldMark first pretrain watermark encoder and decoder, which can minorly adjust protein structures to embed user-specific information and faithfully recover the information from the encoded structure. In the second step, protein generative models are fine-tuned with watermark-conditioned Low-Rank Adaptation (LoRA) modules to preserve generation quality while learning to generate watermarked structures with high recovery rates. Extensive experiments are conducted on open-source protein structure prediction models (e.g., ESMFold and MultiFlow) and de novo structure design models (e.g., FrameDiff and FoldFlow) and we demonstrate that our method is effective across all these generative models. Meanwhile, our watermarking framework only exerts a negligible impact on the original protein structure quality and is robust under potential post-processing and adaptive attacks.

Figures (3)

• Figure 1: Illustrations of application scenarios of FoldMark. The scenario involves the model owner Alice, the thief Bob, and the user Carol. Alice is responsible for training the model, releasing the pretrained model and inference code, or deploying it on the platform for users. Bob, who downloads Alice's model and code, generates protein structures and falsely claims ownership of the copyrights. Carol registers as a user on the server and utilizes the API to generate protein structures. Alice, as the model owner, may wish to restrict the use of these generated structures, particularly in commercial contexts, to avoid copyright infringement. Using FoldMark, Alice can embed a watermark within the generated protein structures, allowing her to extract the watermark code for detection and identification of unauthorized usage, safeguarding her intellectual property.
• Figure 2: Pretraining stage of FoldMark.
• Figure 3: Finetuning stage of FoldMark.