AGATE: Stealthy Black-box Watermarking for Multimodal Model Copyright Protection

TL;DR

The paper tackles copyright protection for multimodal foundation models by proposing AGATE, a black-box watermarking framework that uses in-distribution adversarial triggers and a post-hoc transform module to enable two-phase ownership verification. This approach links watermark provenance to model outputs without requiring fine-tuning or OoD data, preserving downstream performance. The authors introduce a threat-model-driven methodology, a dynamic trigger generation strategy, and a dual-phase verification scheme that jointly enforce trigger ownership and transform-consistency, making forgery difficult. Empirical results on five datasets demonstrate negligible degradation relative to the original models and strong robustness against adversarial attempts, highlighting practical applicability for IP protection in commercial multimodal AI services.

Abstract

Recent advancement in large-scale Artificial Intelligence (AI) models offering multimodal services have become foundational in AI systems, making them prime targets for model theft. Existing methods select Out-of-Distribution (OoD) data as backdoor watermarks and retrain the original model for copyright protection. However, existing methods are susceptible to malicious detection and forgery by adversaries, resulting in watermark evasion. In this work, we propose Model-\underline{ag}nostic Black-box Backdoor W\underline{ate}rmarking Framework (AGATE) to address stealthiness and robustness challenges in multimodal model copyright protection. Specifically, we propose an adversarial trigger generation method to generate stealthy adversarial triggers from ordinary dataset, providing visual fidelity while inducing semantic shifts. To alleviate the issue of anomaly detection among model outputs, we propose a post-transform module to correct the model output by narrowing the distance between adversarial trigger image embedding and text embedding. Subsequently, a two-phase watermark verification is proposed to judge whether the current model infringes by comparing the two results with and without the transform module. Consequently, we consistently outperform state-of-the-art methods across five datasets in the downstream tasks of multimodal image-text retrieval and image classification. Additionally, we validated the robustness of AGATE under two adversarial attack scenarios.

Figures (5)

• Figure 1: Backdoor-based watermarking scheme comparison.
• Figure 2: Framework overview of the proposed AGATE.
• Figure 3: Distributions of image-text pairs in CLIP and transform module embedding space.
• Figure 4: Visualization for different types of trigger generation strategies.
• Figure 5: Performance under different trigger numbers.