Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction

TL;DR

This work investigates privacy risks posed by feature inversion attacks on DNN representations, especially in split DNN setups. It introduces diffusion-model priors, via latent diffusion models, to invert intermediate features, and augments them with textual priors and cross-frame temporal coherence to boost reconstruction quality. The authors demonstrate that diffusion-prior-based inversion significantly outperforms direct optimization and decoder baselines across image classification, object detection, and multimodal CLIP backbones, in both white-box and black-box scenarios, and show gains from text and temporal priors. They also discuss practical defense options (MPC, HE, differential privacy) and emphasize that deeper models do not inherently guarantee privacy, prompting further study of robust safeguards. Overall, the paper reveals strong privacy leakage through generative priors and lays groundwork for both advancing inversion-attacks and informing defenses in real-world DNN feature-based systems.

Abstract

Inverting visual representations within deep neural networks (DNNs) presents a challenging and important problem in the field of security and privacy for deep learning. The main goal is to invert the features of an unidentified target image generated by a pre-trained DNN, aiming to reconstruct the original image. Feature inversion holds particular significance in understanding the privacy leakage inherent in contemporary split DNN execution techniques, as well as in various applications based on the extracted DNN features. In this paper, we explore the use of diffusion models, a promising technique for image synthesis, to enhance feature inversion quality. We also investigate the potential of incorporating alternative forms of prior knowledge, such as textual prompts and cross-frame temporal correlations, to further improve the quality of inverted features. Our findings reveal that diffusion models can effectively leverage hidden information from the DNN features, resulting in superior reconstruction performance compared to previous methods. This research offers valuable insights into how diffusion models can enhance privacy and security within applications that are reliant on DNN features.

Figures (25)

• Figure 1: The DNN undergoes layer-wise partitioning and is divided between the edge and the cloud, with the intermediate features being transferred between them. In this example, we focus on the presence of a single edge device.
• Figure 2: Feature inversion attack against Split DNN execution with white-box settings. $F_{1}(.)$ is exposed to the attacker.
• Figure 3: Black-box feature inversion attack procedures.
• Figure 4: Feature inversion with diffusion model.
• Figure 5: Impact of textual prior on feature inversion.