Detecting Backdoor Samples in Contrastive Language Image Pretraining

TL;DR

CLIP pretraining on web-scale data is vulnerable to backdoor poisoning at rates as low as $0.01\%$. The authors show backdoor samples in CLIP representations form sparse local neighborhoods, enabling highly effective detection by simple density-based outlier methods (SLOF, DAO, k-dist) that outperform existing supervised detectors. They demonstrate practical purification of million-scale datasets in about 15 minutes on 4 Nvidia A100 GPUs and reveal unintentional backdoors in OpenCLIP-trained models through trigger recovery. The work emphasizes the necessity of data curation for large-scale multimodal systems and provides scalable tooling for defending CLIP-style pretraining.

Abstract

Contrastive language-image pretraining (CLIP) has been found to be vulnerable to poisoning backdoor attacks where the adversary can achieve an almost perfect attack success rate on CLIP models by poisoning only 0.01\% of the training dataset. This raises security concerns on the current practice of pretraining large-scale models on unscrutinized web data using CLIP. In this work, we analyze the representations of backdoor-poisoned samples learned by CLIP models and find that they exhibit unique characteristics in their local subspace, i.e., their local neighborhoods are far more sparse than that of clean samples. Based on this finding, we conduct a systematic study on detecting CLIP backdoor attacks and show that these attacks can be easily and efficiently detected by traditional density ratio-based local outlier detectors, whereas existing backdoor sample detection methods fail. Our experiments also reveal that an unintentional backdoor already exists in the original CC3M dataset and has been trained into a popular open-source model released by OpenCLIP. Based on our detector, one can clean up a million-scale web dataset (e.g., CC3M) efficiently within 15 minutes using 4 Nvidia A100 GPUs. The code is publicly available in our \href{https://github.com/HanxunH/Detect-CLIP-Backdoor-Samples}{GitHub repository}.

Figures (16)

• Figure 1: (a) The CLIP learned presentations are projected into a 2-D space using t-SNE. The red cross is a backdoor data point, the green dot is a clean data point, and the blue dot is a randomly sampled data point. The $\mathop{\mathrm{\mathit{k}\textrm{-dist}}}\nolimits$ is the distance to the $k$-th nearest neighbor, and the circle with the solid line is the region containing all $k$ nearest neighbors. The circle with a dashed line is the region containing $k$ nearest neighbors for the $k$-th neighbors. $k$ is set to 16. (b) The distribution $\mathop{\mathrm{\mathit{k}\textrm{-dist}}}\nolimits$ for clean and backdoor data with different poisoning rates (PR) within a batch.
• Figure 2: The distributions of the DAO detection score on poisoned CC3M using ResNet-50 as the vision encoder.
• Figure 3: (a) Defence performance for varying filtering percentages. (b) An example of the unavailable images. (c-d) The recovered trigger pattern of the birthday cake image on our pre-trained CLIP (b) and a model (c) released by OpenCLIP that uses ResNet-50 as the vision encoder.
• Figure 4: Examples of the 5 different triggers used in the experiments. The patch and clean label attacks use a 16 by 16 patch as the trigger. The clean label attack only applies the trigger to images with captions that contain the keyword specified by the adversary. The Nashville converts the image using the filter template "Nashville." The WaNet applies grided noise to the image. The SIG uses a periodical pattern as the trigger. The blend attack creates an overly transparent Hello Kitty image.
• Figure 5: The detection AUC (%) of different local outlier methods under varying locality $k$. The batch size is set to 2048 for all experiments.

Theorems & Definitions (1)

• Theorem 1: houle2017local1