Reconstructing Training Data From Real World Models Trained with Transfer Learning

TL;DR

This work investigates privacy risks from training-data reconstruction in realistic transfer-learning scenarios that use embeddings from large foundation models. It introduces a two-stage approach: first reconstruct embedding vectors from a classifier trained on these embeddings, then map those embeddings back to images via model inversion, using a Deep Image Prior for most backbones and UnCLIP for CLIP. A key contribution is a clustering-based method to identify good reconstructions without access to the original training set, enabling practical attacks on Food-101 and iNaturalist across multiple backbones. The findings reveal a tangible privacy threat in transfer-learning pipelines and discuss limitations and potential defenses, emphasizing the need for defenses as transfer learning remains widespread.

Abstract

Current methods for reconstructing training data from trained classifiers are restricted to very small models, limited training set sizes, and low-resolution images. Such restrictions hinder their applicability to real-world scenarios. In this paper, we present a novel approach enabling data reconstruction in realistic settings for models trained on high-resolution images. Our method adapts the reconstruction scheme of arXiv:2206.07758 to real-world scenarios -- specifically, targeting models trained via transfer learning over image embeddings of large pre-trained models like DINO-ViT and CLIP. Our work employs data reconstruction in the embedding space rather than in the image space, showcasing its applicability beyond visual data. Moreover, we introduce a novel clustering-based method to identify good reconstructions from thousands of candidates. This significantly improves on previous works that relied on knowledge of the training set to identify good reconstructed images. Our findings shed light on a potential privacy risk for data leakage from models trained using transfer learning.

Figures (29)

• Figure 1: Reconstructed Data from a binary classifier trained on $100$ DINO-VIT embeddings
• Figure 2: Overview of our training and data reconstruction scheme.
• Figure 3: Training samples (red) and their best reconstructed candidate, from MLPs trained on embeddings of various backbone models for two datasets.
• Figure 4: Reconstructions from a multiclass models trained $100$ images from Food101/iNaturalist with $C$=$10/4$ classes ($10/25$ images per class), with test-accuracy $84\%$/$96\%$ (on a/b respectively). Color-padded images are training images, where color represents different classes.
• Figure 5: Numerical summary for all models whose reconstructed samples are shown in \ref{['fig:main_results', 'fig:multiclass_10']}.