Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Services

TL;DR

This work formalizes Pre-trained Encoder Inference (PEI), a black-box attack that can deduce which hidden pre-trained encoder powers a downstream service using only API access and a set of candidate encoders. It introduces a two-stage framework that synthesizes PEI attack samples via zeroth-order gradient estimation and then uses a z-test on behavior similarity to infer the hidden encoder, achieving practical efficiency and low false-positive rates. Empirically, PEI effectively reveals encoders in vision-based image classification and LLaVA multimodal generation services and demonstrates that revealed encoders significantly enhance downstream attacks such as model stealing and adversarial manipulation. The findings underscore important security and privacy implications for encoder-based services and motivate defenses against downstream-side threats in real-world EaaS deployments.

Abstract

Pre-trained encoders available online have been widely adopted to build downstream machine learning (ML) services, but various attacks against these encoders also post security and privacy threats toward such a downstream ML service paradigm. We unveil a new vulnerability: the Pre-trained Encoder Inference (PEI) attack, which can extract sensitive encoder information from a targeted downstream ML service that can then be used to promote other ML attacks against the targeted service. By only providing API accesses to a targeted downstream service and a set of candidate encoders, the PEI attack can successfully infer which encoder is secretly used by the targeted service based on candidate ones. Compared with existing encoder attacks, which mainly target encoders on the upstream side, the PEI attack can compromise encoders even after they have been deployed and hidden in downstream ML services, which makes it a more realistic threat. We empirically verify the effectiveness of the PEI attack on vision encoders. we first conduct PEI attacks against two downstream services (i.e., image classification and multimodal generation), and then show how PEI attacks can facilitate other ML attacks (i.e., model stealing attacks vs. image classification models and adversarial attacks vs. multimodal generative models). Our results call for new security and privacy considerations when deploying encoders in downstream services. The code is available at https://github.com/fshp971/encoder-inference.

Figures (8)

• Figure 1: Illustration of how the PEI attack can threaten downstream ML services. Step 1: Using the PEI attack to reveal the encoder hidden in the targeted downstream service. Step 2: Exploiting the revealed encoder to conduct other ML attacks, e.g., model stealing and adversarial attacks, against the original targeted service.
• Figure 2: PEI $z$-scores of candidates on different CIFAR-10 classification services where the correct hidden encoder is not in the PEI candidates set. $z$-scores that are above the threshold $1.7$ are highlighted. Ideally, none of the reported $z$-score should go beyond the preset threshold.
• Figure 3: Two examples of adversarial attacks against LLaVA with adversarial images synthesized based on the hidden vision encoder revealed by the PEI attack. These adversarial images contain visually benign mosaics, but can induce LLaVA to generate predefined false health/medical information.
• Figure 4: The $10$ objective images used in the PEI attack against vision encoders.
• Figure 5: Rates of different PEI attack results on CIFAR-10, SVHN, and Food-101 datasets. Rates of attack success, false-negative, and false-positive are colored differently.