Semantic Deep Hiding for Robust Unlearnable Examples

TL;DR

A Deep Hiding scheme that adaptively hides semantic images enriched with high-level features, employing an Invertible Neural Network to invisibly integrate predefined images, inherently hiding them with deceptive perturbations is proposed.

Abstract

Ensuring data privacy and protection has become paramount in the era of deep learning. Unlearnable examples are proposed to mislead the deep learning models and prevent data from unauthorized exploration by adding small perturbations to data. However, such perturbations (e.g., noise, texture, color change) predominantly impact low-level features, making them vulnerable to common countermeasures. In contrast, semantic images with intricate shapes have a wealth of high-level features, making them more resilient to countermeasures and potential for producing robust unlearnable examples. In this paper, we propose a Deep Hiding (DH) scheme that adaptively hides semantic images enriched with high-level features. We employ an Invertible Neural Network (INN) to invisibly integrate predefined images, inherently hiding them with deceptive perturbations. To enhance data unlearnability, we introduce a Latent Feature Concentration module, designed to work with the INN, regularizing the intra-class variance of these perturbations. To further boost the robustness of unlearnable examples, we design a Semantic Images Generation module that produces hidden semantic images. By utilizing similar semantic information, this module generates similar semantic images for samples within the same classes, thereby enlarging the inter-class distance and narrowing the intra-class distance. Extensive experiments on CIFAR-10, CIFAR-100, and an ImageNet subset, against 18 countermeasures, reveal that our proposed method exhibits outstanding robustness for unlearnable examples, demonstrating its efficacy in preventing unauthorized data exploitation.

Figures (6)

• Figure 1: An illustration depicts the concept of unlearnable examples, where semantic images are cleverly embedded within clean images to create these unlearnable examples. When Deep Neural Networks (DNNs) are trained on these examples, they fail to learn the meaningful features of the clean images due to the embedded semantic perturbations. Consequently, the accuracy of the DNN models on clean data becomes significantly and unpredictably poor.
• Figure 2: Overall pipeline of the proposed method. A generative model is employed to generate the hidden semantic images. These generated images are then hidden within clean images using a Deep Hiding model. The Latent Feature Concentration module is designed to constrain the intra-class variance by regularizing the latent feature distance of perturbations.
• Figure 3: Grad-CAM visualization of unlearnable examples derived from the ImageNet subset under different countermeasures. Note that red regions typically indicate the areas the model paid the most attention to, while Blue regions colors indicate less attention.
• Figure 4: The t-SNE visualization of the models' feature representations on the clean samples (left) and the perturbation generated by our DH scheme (right) on CIFAR-10.
• Figure 5: The visualization of the unlearnable examples generated by different methods is shown in columns 2-5, where the second and fourth rows correspond to the perturbation maps. Perturbations are absoluted and normalized to [0,1] for a better view.