Do Vision-Language Models Leak What They Learn? Adaptive Token-Weighted Model Inversion Attacks

TL;DR

<3-5 sentence high-level summary> This paper provides the first systematic exploration of model inversion attacks on Vision-Language Models (VLMs), showing that VLMs leak private training visual data through their token-based and sequence-based outputs. It introduces four MI strategies (TMI, TMI-C, SMI, SMI-AW) with SMI-AW dynamically weighting tokens by visual grounding to produce more informative gradients. Across multiple state-of-the-art and publicly released VLMs, the attacks achieve substantial privacy leakage, achieving up to 61.21% human-evaluated attack success and high identity reconstruction quality. The work highlights urgent privacy concerns and the need for safeguards as VLMs are deployed in sensitive domains.

Abstract

Model inversion (MI) attacks pose significant privacy risks by reconstructing private training data from trained neural networks. While prior studies have primarily examined unimodal deep networks, the vulnerability of vision-language models (VLMs) remains largely unexplored. In this work, we present the first systematic study of MI attacks on VLMs to understand their susceptibility to leaking private visual training data. Our work makes two main contributions. First, tailored to the token-generative nature of VLMs, we introduce a suite of token-based and sequence-based model inversion strategies, providing a comprehensive analysis of VLMs' vulnerability under different attack formulations. Second, based on the observation that tokens vary in their visual grounding, and hence their gradients differ in informativeness for image reconstruction, we propose Sequence-based Model Inversion with Adaptive Token Weighting (SMI-AW) as a novel MI for VLMs. SMI-AW dynamically reweights each token's loss gradient according to its visual grounding, enabling the optimization to focus on visually informative tokens and more effectively guide the reconstruction of private images. Through extensive experiments and human evaluations on a range of state-of-the-art VLMs across multiple datasets, we show that VLMs are susceptible to training data leakage. Human evaluation of the reconstructed images yields an attack accuracy of 61.21%, underscoring the severity of these privacy risks. Notably, we demonstrate that publicly released VLMs are vulnerable to such attacks. Our study highlights the urgent need for privacy safeguards as VLMs become increasingly deployed in sensitive domains such as healthcare and finance. Additional experiments are provided in Supp.

Figures (18)

• Figure 1: We conduct the first systematic study of MI attacks on VLMs.(A) Designed for the token-generative characteristics of VLMs, we introduce a set of token-level and sequence-level MI strategies to investigate VLMs' privacy vulnerability (Sec. \ref{['sec:MI']}). Particularly, conventional MI typically targets unimodal DNNs, where the adversary seeks to reconstruct a training image $x = G(w)$ that maximizes the likelihood of a target class label $y$ under the target model $M_{DNN}$ by repeating $N$ inversion steps. In contrast, VLMs $M_{VLM}$ generate a sequence of tokens, and the target output $\mathbf{y} = (y_1, \dots, y_m)$ is also a sequence of $m$ tokens. To address the unique nature of VLMs, we introduce several MI strategies: Token-based Model Inversion (TMI), Convergent Token-based Model Inversion (TMI-C), and Sequence-based Model Inversion (SMI). (B) Building on the insight that output tokens differ in their degree of visual grounding, and hence their gradients vary in informativeness for reconstructing images during inversion, we propose Sequence-based Model Inversion with Adaptive Token Weighting (SMI-AW), a novel MI for VLMs (Sec. \ref{['sec:SMI-AW']}). SMI-AW adaptively adjusts each token's gradient contribution according to its visual grounding, allowing the optimization to concentrate on visually grounded tokens and more effectively recover private training images. See Figure \ref{['fig:attentionMap']} for discussion of attention map analysis.
• Figure 2: Analysis of visual–textual attention across output tokens and inversion steps. We visualize the cross-attention map between the reconstructed image and each output token during inversion. Different tokens exhibit markedly different attention maps: visually grounded tokens show strong attention, while others produce weak responses, indicating limited reliance on the image. Moreover, attention patterns evolve over inversion steps, as a token's dependence on visual input changes when the reconstructed image becomes more consistent with the target output. These observations reveal that token-level gradients vary substantially in visual informativeness both across tokens and over time. This motivates our SMI-AW method, which dynamically reweights token contributions based on their visual attention strength. Additional attention map analysis can be found in Supp.
• Figure 3: The match rate between the output text of the reconstructed image and the target output text $y$.
• Figure 4: Qualitative results on the Facescrub dataset using LLaVA-v1.6-7B model with our SMI-AW and $\mathcal{L}_{LOM}$. The first row shows images from the private training dataset, while the second row presents the reconstructed images corresponding to each individual in the first row. The visual similarity between the original and reconstructed images demonstrates the effectiveness of our inversion method in recovering private training data. More reconstructed images can be found in Supp.
• Figure 5: We reconstruct images of celebrities from the pre-trained LLaVA-v1.6-7B model. We use SMI-AW with $\mathcal{L}_{LOM}$ to reconstruct images. For each pair, the left image shows a training image in $\mathcal{D}_{priv}$, while the right image presents the reconstruction $x_{recon}$ obtained via our model inversion attack. This result illustrates that the pre-trained VLM is vulnerable to training data leakage through model inversion. More results can be found in Supp.