Adaptive Domain Inference Attack with Concept Hierarchy

TL;DR

This work studies whether removing domain information from model APIs can protect against privacy attacks like model inversion, showing that attackers can still infer target-domain concepts using a black-box interface. It introduces Adaptive Domain Inference (ADI), which leverages a hierarchical concept structure to adaptively bias sampling toward leaf concepts likely present in the target domain, reducing the number of API accesses needed. ADI outperforms prior attempts (GDI and LDI) in both the quality of inferred data and efficiency, demonstrated across synthesized and ImageNet-related datasets via OTDD metrics and downstream model-inversion enhancements. The findings highlight persistent privacy risks in API-based model deployment and motivate more robust protection mechanisms beyond domain-agnostic API exposure.

Abstract

With increasingly deployed deep neural networks in sensitive application domains, such as healthcare and security, it's essential to understand what kind of sensitive information can be inferred from these models. Most known model-targeted attacks assume attackers have learned the application domain or training data distribution to ensure successful attacks. Can removing the domain information from model APIs protect models from these attacks? This paper studies this critical problem. Unfortunately, even with minimal knowledge, i.e., accessing the model as an unnamed function without leaking the meaning of input and output, the proposed adaptive domain inference attack (ADI) can still successfully estimate relevant subsets of training data. We show that the extracted relevant data can significantly improve, for instance, the performance of model-inversion attacks. Specifically, the ADI method utilizes a concept hierarchy extracted from a collection of available public and private datasets and a novel algorithm to adaptively tune the likelihood of leaf concepts showing up in the unseen training data. We also designed a straightforward hypothesis-testing-based attack -- LDI. The ADI attack not only extracts partial training data at the concept level but also converges fastest and requires the fewest target-model accesses among all candidate methods. Our code is available at https://anonymous.4open.science/r/KDD-362D.

Figures (13)

• Figure 1: Concept hierarchy illustration. The $i$-th node at level $j$ holds probability $p_{ij}^{(t)}$ at time step $t$. Initially, node probabilities are set to $1/q$, with $q$ as the number of child nodes under the node's parent. In each iteration, Algorithm \ref{['alg:general']} applies a random walk from the root to a leaf, e.g., a red path. Then, a sample image is drawn from the leaf node and applied to the target model. The result will trigger the probability updates following a green path from the leaf to the root.
• Figure 2: Comparison of performance on high-resolution datasets and large-scale concept hierarchy. ADI outperforms other domain inference attacks in both efficiency and efficacy.
• Figure 3: OTDD constantly decreases with the increasing size of tested samples in LDI. Dataset Names: M-MNIST, E-EMNIST, L-LFW, C-CIFAR, CN-CINIC, FM-FashionMNIST, CL-CLOTHING, NETTE-ImageNETTE, Woof-ImageWoof, Fashion-DeepFashion.
• Figure 4: Examining the effects of $\delta$ configurations.
• Figure 5: Comparison of entropy-based and LiRA-based methods with and without rebalancing on different datasets.