DiMEx: Breaking the Cold Start Barrier in Data-Free Model Extraction via Latent Diffusion Priors

TL;DR

DiMEx eliminates the Cold Start barrier in data-free model extraction by leveraging latent diffusion priors and latent-space Bayesian optimization, enabling semantically valid queries from the first iteration and achieving high-audience agreement with as few as 2,000 queries. It replaces untrained GANs with a frozen diffusion model (Stable Diffusion) and optimizes in the latent space via REMBO, augmented by vicinal sampling to capture local boundary structure. To counter this semantic threat, the Hybrid Stateful Ensemble (HSE) defense combines spatial multi-model consensus with temporal latent-drift analysis to detect and suppress optimization-driven attacks with low latency and minimal impact on legitimate users. The results show strong cold-start performance across multiple datasets, while HSE achieves substantial reductions in attack success rates, highlighting the need for stateful, trajectory-aware defenses in modern MLaaS deployments saturated by diffusion-based priors.

Abstract

Model stealing attacks pose an existential threat to Machine Learning as a Service (MLaaS), allowing adversaries to replicate proprietary models for a fraction of their training cost. While Data-Free Model Extraction (DFME) has emerged as a stealthy vector, it remains fundamentally constrained by the "Cold Start" problem: GAN-based adversaries waste thousands of queries converging from random noise to meaningful data. We propose DiMEx, a framework that weaponizes the rich semantic priors of pre-trained Latent Diffusion Models to bypass this initialization barrier entirely. By employing Random Embedding Bayesian Optimization (REMBO) within the generator's latent space, DiMEx synthesizes high-fidelity queries immediately, achieving 52.1 percent agreement on SVHN with just 2,000 queries - outperforming state-of-the-art GAN baselines by over 16 percent. To counter this highly semantic threat, we introduce the Hybrid Stateful Ensemble (HSE) defense, which identifies the unique "optimization trajectory" of latent-space attacks. Our results demonstrate that while DiMEx evades static distribution detectors, HSE exploits this temporal signature to suppress attack success rates to 21.6 percent with negligible latency.

Figures (3)

• Figure 1: DiMEx System Architecture. The attack utilizes a Random Embedding Bayesian Optimization (REMBO) module to search the low-dimensional latent space of a pre-trained Diffusion Generator. A "Vicinal Sampling" block generates clouds of perturbed latent vectors to robustly estimate gradients from the black-box Victim Oracle, overcoming the cold-start problem inherent in traditional GAN-based attacks.
• Figure 2: Qualitative Comparison of Query Generation Phases.Row 1 (Victim Data): Ground truth samples from SVHN showing clear semantic structure. Row 2 (Cold Start): Standard GAN initialization produces high-frequency Gaussian noise, yielding vanishing gradients ($\nabla \approx 0$) and wasting the first $\sim$2k queries. Row 3 (BESA ren2025besa): Feature recovery attacks often introduce high-frequency artifacts (grid-like noise) that can be filtered by defenses. Row 4 (DiMEx - Ours): By optimizing in the latent space of a frozen Stable Diffusion prior, DiMEx generates semantically valid digit-like structures immediately (Iteration 0). This ensures the surrogate model receives informative hard-label supervision from the very first query, bypassing the Cold Start barrier.
• Figure 3: Visual Comparison. DiMEx (Red) demonstrates rapid convergence in the critical "Cold Start" phase ($<10k$ queries), particularly on high-complexity datasets like STL-10 and CIFAR-100 where GAN-based methods (Orange) fail to generate meaningful queries.