Three-in-One: Robust Enhanced Universal Transferable Anti-Facial Retrieval in Online Social Networks

TL;DR

The paper tackles the privacy risk of deep hash-based facial retrieval in OSNs by introducing TOAP, a robust universal adversarial perturbation. TOAP combines a local and global Compression Generator to emulate OSN post-processing, and uses a meta-learning based optimization from point to space against cluster and data-space centers, with pixel-, feature-, and hash-level fine-tuning losses. The approach yields superior universality, transferability, and robustness across multiple datasets, hash models, and real OSNs, outperforming state-of-the-art methods and demonstrating practical privacy protection for users. The work advances privacy-preserving retrieval in real-world social platforms and provides a framework for evaluating adversarial robustness under complex post-processing pipelines.

Abstract

Deep hash-based retrieval techniques are widely used in facial retrieval systems to improve the efficiency of facial matching. However, it also carries the danger of exposing private information. Deep hash models are easily influenced by adversarial examples, which can be leveraged to protect private images from malicious retrieval. The existing adversarial example methods against deep hash models focus on universality and transferability, lacking the research on its robustness in online social networks (OSNs), which leads to their failure in anti-retrieval after post-processing. Therefore, we provide the first in-depth discussion on robustness adversarial perturbation in universal transferable anti-facial retrieval and propose Three-in-One Adversarial Perturbation (TOAP). Specifically, we construct a local and global Compression Generator (CG) to simulate complex post-processing scenarios, which can be used to mitigate perturbation. Then, we propose robust optimization objectives based on the discovery of the variation patterns of model's distribution after post-processing, and generate adversarial examples using these objectives and meta-learning. Finally, we iteratively optimize perturbation by alternately generating adversarial examples and fine-tuning the CG, balancing the performance of perturbation while enhancing CG's ability to mitigate them. Numerous experiments demonstrate that, in addition to its advantages in universality and transferability, TOAP significantly outperforms current state-of-the-art methods in multiple robustness metrics. It further improves universality and transferability by 5% to 28%, and achieves up to about 33% significant improvement in several simulated post-processing scenarios as well as mainstream OSNs, demonstrating that TOAP can effectively protect private images from malicious retrieval in real-world scenarios.

Figures (15)

• Figure 1: Threat Scenario. The red arrows indicate that adversary utilizes a photo containing user’s identity to retrieve user’s relevant photos, which leads to privacy leakage, such as user's daily life, job, and health condition. The green arrows indicate that adversary is unable to retrieve user’s relevant photos using user’s photo with robust perturbation, thus protecting user’s privacy.
• Figure 2: Visualization of the change in model's focus areas before (line 1) and after (line 2) JPEG compression (quality factor is 60). The experimental setting is CASIA, DHD, VGG16.
• Figure 3: The pipeline of TOAP. a) using local and global CG to process adversarial examples; b) optimizing perturbation from point to space using meta-learning; c) fine-tuning CG using pixel-, feature, and hash-level losses.
• Figure 4: The pipeline of local and global CG.
• Figure 5: The relationship between the number of database samples and the Hamming distance between: (a) samples and their cluster centers; (b) original samples and post-processed samples. Y-axis denotes the number of samples and X-axis denotes the Hamming distance. The experimental setting is CASIA, DHD, ResNet50. The quality factor of JPEG compression is 60, the resize ratio is 3/2, and the Gaussian kernel size for Gaussian blur is 3 with a standard deviation of 0.8.