Revisiting Model Inversion Evaluation: From Misleading Standards to Reliable Privacy Assessment

TL;DR

This work challenges the reliability of the standard Model Inversion evaluation framework $F_{Curr}$, which can yield inflated attack success due to Type I adversarial features that do not reflect true visual identity. It establishes a formal link between MI false positives and Type I adversarial examples, and demonstrates that adversarial transferability can cause these false positives to transfer to the evaluation model $E$. To remedy this, it introduces $F_{MLLM}$, an evaluation framework based on Multimodal Large Language Models that reduces transferability-driven false positives by avoiding reliance on the same task design as the target model. Across 27 MI setups, the authors show that $F_{Curr}$ can report AttAcc up to near 100% while actual leakage assessed by $F_{MLLM}$ remains substantially lower, underscoring the need for a more faithful privacy assessment. The proposed framework, together with released code, provides a robust new standard for evaluating privacy risks in ML systems and calls for reevaluation of reported progress in MI research.

Abstract

Model Inversion (MI) attacks aim to reconstruct information from private training data by exploiting access to machine learning models T. To evaluate such attacks, the standard evaluation framework relies on an evaluation model E, trained under the same task design as T. This framework has become the de facto standard for assessing progress in MI research, used across nearly all recent MI studies without question. In this paper, we present the first in-depth study of this evaluation framework. In particular, we identify a critical issue of this standard framework: Type-I adversarial examples. These are reconstructions that do not capture the visual features of private training data, yet are still deemed successful by T and ultimately transferable to E. Such false positives undermine the reliability of the standard MI evaluation framework. To address this issue, we introduce a new MI evaluation framework that replaces the evaluation model E with advanced Multimodal Large Language Models (MLLMs). By leveraging their general-purpose visual understanding, our MLLM-based framework does not depend on training of shared task design as in T, thus reducing Type-I transferability and providing more faithful assessments of reconstruction success. Using our MLLM-based evaluation framework, we reevaluate 27 diverse MI attack setups and empirically reveal consistently high false positive rates under the standard evaluation framework. Importantly, we demonstrate that many state-of-the-art (SOTA) MI methods report inflated attack accuracy, indicating that actual privacy leakage is significantly lower than previously believed. By uncovering this critical issue and proposing a robust solution, our work enables a reassessment of progress in MI research and sets a new standard for reliable and robust evaluation. Code can be found in https://github.com/hosytuyen/MI-Eval-MLLM

Figures (2)

• Figure 1: We present the first and in-depth study on the Model Inversion (MI) evaluation. Particularly, we investigate the most common MI evaluation framework $\mathcal{F}_{Curr}$ to measure MI Attack Accuracy (AttAcc). $\mathcal{F}_{Curr}$ is introduced in zhang2020secret and is utilized to assess almost all recent MI attacks/defenses. However, we find that $\mathcal{F}_{Curr}$ suffers from a significant number of false positives. These false positive MI reconstructed samples do not capture visual identity features of the target individual in the private training data, but they are still deemed successful attacks according to $\mathcal{F}_{Curr}$ with a high confidence (indicated in red text). Extensive false positives are in the Supp.
• Figure 2: An example of evaluation query in our F_MLLM. The task is to determine whether "Image A" depicts the same individual as those in "Image B". We have two setups: (1) "Image A" and "Image B" consist of private images and (2) "Image A" is an MI-reconstructed image $x^{r}_{y}$ of the target label $y$ while four real images of $y$ are randomly selected as "Image B". MLLM is tasked with responding either "Yes" or "No" to indicate whether "Image A" matches the identity in "Image B".