Captured by Captions: On Memorization and its Mitigation in CLIP Models

TL;DR

This paper defines CLIPMem, a memorization metric for vision-language CLIP models, and demonstrates that memorization in CLIP sits between supervised and self-supervised paradigms, with mis-captioned and atypical data driving strongest memorization. By analyzing both modalities jointly and using a leave-one-out comparison between a model trained with and without the data point, CLIPMem reveals that the text encoder is the major contributor to memorization. The authors show that text-centric augmentations, multiple captions, and embedding-noise strategies can reduce memorization while simultaneously improving downstream generalization, challenging the usual trade-off observed in single-modal memorization. The work also finds that memorization signals amplify privacy and data-noise concerns in web-sourced data and provides practical mitigation paths for more robust CLIP training and data curation. Overall, CLIPMem offers a principled, actionable framework to quantify and mitigate memorization in multimodal representations, with implications for privacy, data curation, and model utility.

Abstract

Multi-modal models, such as CLIP, have demonstrated strong performance in aligning visual and textual representations, excelling in tasks like image retrieval and zero-shot classification. Despite this success, the mechanisms by which these models utilize training data, particularly the role of memorization, remain unclear. In uni-modal models, both supervised and self-supervised, memorization has been shown to be essential for generalization. However, it is not well understood how these findings would apply to CLIP, which incorporates elements from both supervised learning via captions that provide a supervisory signal similar to labels, and from self-supervised learning via the contrastive objective. To bridge this gap in understanding, we propose a formal definition of memorization in CLIP (CLIPMem) and use it to quantify memorization in CLIP models. Our results indicate that CLIP's memorization behavior falls between the supervised and self-supervised paradigms, with "mis-captioned" samples exhibiting highest levels of memorization. Additionally, we find that the text encoder contributes more to memorization than the image encoder, suggesting that mitigation strategies should focus on the text domain. Building on these insights, we propose multiple strategies to reduce memorization while at the same time improving utility--something that had not been shown before for traditional learning paradigms where reducing memorization typically results in utility decrease.

Figures (19)

• Figure 1: Examples of data with different levels of memorization. Higher memorization scores indicate stronger memorization. We observe that atypical or distorted images, as well as those with incorrect or imprecise captions, experience higher memorization compared to standard samples and easy-to-label images with accurate captions. Results are obtained on OpenCLIP ilharco_gabriel_2021_5143773, with encoders based on the ViT-Base architecture trained on the COCO dataset.
• Figure 2: Memorization with CLIPMem. We train a CLIP model on COCO using standard image cropping and no text augmentations. (a) We present memorization scores according to CLIPMem per data subset. The significantly higher scores for $S_C$ compared to $S_S$ indicate that $f$ memorizes $S_C$. (b) We also study how inserting training samples with imprecise or incorrect captions ("mis-captioned") affects memorization. We refer to the model trained with correct captions as Clean Model, and the model trained with $S_C$ containing 4500 standard canaries (Clean) and 500 mis-captioned (Mis-captioned) as Poisoned Model. We report CLIPMem over the different subsets of candidates. We observe that the mis-captioned samples experience a significantly higher memorization while the memorization of the clean data points is (almost) not affected.
• Figure 3: Measuring memorization on individual modalities is not able to extract a strong signal. (a)--(b) We measure SSLMem wang2024memorization on the individual encoders of our CLIP model trained on COCO. (c) Our CLIPMem extracts a stronger memorization signal by using both modalities in CLIP jointly.
• Figure 4: UnitMem metric: CLIP is between supervised and SSL models.
• Figure 5: Mitigating memorization in CLIP improves downstream generalization. We train CLIP models with different "augmentations" in the textual domain. (a) We use multiple captions for the same image during training. (b) We directly noise the text embeddings during the training using Gaussian noise with a mean of 0 and different standard deviations (adding the Gaussian noise $\mathcal{N}(0,0.15)$ gives us the sweet spot with the smallest memorization and highest performance). Both strategies successfully reduce memorization while improving performance.