From Head to Tail: Efficient Black-box Model Inversion Attack via Long-tailed Learning

TL;DR

The paper tackles the privacy risk of model inversion attacks in black-box settings by proposing SMILE, a two-step framework that combines long-tailed surrogate training with NGOpt-guided gradient-free optimization to reconstruct target identities from GAN-generated samples. Key innovations include a long-tail aware surrogate training process with an ensemble, distillation, diversity regularization, and Top-k reweighting, followed by a constrained latent-space search using NGOpt and P-space clipping. Empirical results show SMILE outperforms state-of-the-art black-box MIAs across multiple datasets and target models while reducing query overhead to roughly 5% of prior methods, and robustness is demonstrated through ablation and defense analyses. The work underscores persistent privacy risks in black-box settings and highlights how initialization quality and problem-specific optimization strategies critically shape attack practicality and efficiency.

Abstract

Model Inversion Attacks (MIAs) aim to reconstruct private training data from models, leading to privacy leakage, particularly in facial recognition systems. Although many studies have enhanced the effectiveness of white-box MIAs, less attention has been paid to improving efficiency and utility under limited attacker capabilities. Existing black-box MIAs necessitate an impractical number of queries, incurring significant overhead. Therefore, we analyze the limitations of existing MIAs and introduce Surrogate Model-based Inversion with Long-tailed Enhancement (SMILE), a high-resolution oriented and query-efficient MIA for the black-box setting. We begin by analyzing the initialization of MIAs from a data distribution perspective and propose a long-tailed surrogate training method to obtain high-quality initial points. We then enhance the attack's effectiveness by employing the gradient-free black-box optimization algorithm selected by NGOpt. Our experiments show that SMILE outperforms existing state-of-the-art black-box MIAs while requiring only about 5% of the query overhead.

Figures (13)

• Figure 1: Visualization of sample distribution across classes.$N$ denotes the number of samples in a single label. The colors of the bar chart indicate the GAN models used for sampling. From (a), it can be seen that with the same image prior, doubling the sampling causes a significant increase in the variance of $N$, but the proportion of $N=0$ only slightly decreases. This indicates that the long-tail distribution is not mitigated. It is also evident that classes with extremely high sample counts at $20K$ sampling remain so at $40K$ sampling, demonstrating the inefficiency of blindly increasing the sampling size. In (b), it can be observed that the bar chart obtained from FFHQ sampling is flatter than that of CelebA, indicating that using a more diverse image prior can effectively mitigate the long-tail distribution, specifically reflected in the smaller variance of $N$. In (c), using a stronger GAN under the same image priors provides only minimal improvement for the long-tail distribution, as evidenced by a slight reduction in variance. Interestingly, classes that occupy more samples under the weaker GAN sampling often also appear in the strong GAN sampling, manifesting as symmetry in the bar charts above and below. However, charts derived from different prior samplings tend to be asymmetrical, as shown in (b). This suggests that the mitigation of long-tail distributions by image priors primarily depends on the data distribution, rather than the advancement of the generative model.
• Figure 1: Distribution of Top-1 confidence scores. The target model is set to VGGFace2,ResNet50.
• Figure 2: The overall architecture of SMILE.
• Figure 2: Model 1 is from Base, Model 2 is from long-tailed surrogate training. The figure shows the model's performance on the first 100 IDs.
• Figure 3: We visualized the optimization processes under different initializations.$w_0$ is the sample with the highest confidence for the target ID in the sample pool, and $w'$ is the sample obtained from the white-box MIA on $M_s$. As shown in (a), long-tailed surrogate training provides a more helpful initial point for black-box optimization, allowing it to approach the target point in very few iterative steps while avoiding local optima. (b) shows that long-tailed surrogate training fully captures the information of this ID, thereby providing a high-quality initial point close to the target point.