Guarding the Privacy of Label-Only Access to Neural Network Classifiers via iDP Verification

TL;DR

This work introduces LUCID, a system that delivers label-only access to neural classifiers under Individual Differential Privacy (iDP) with minimal accuracy loss. The key idea is the iDP-DB, a deterministic overapproximation that identifies inputs unlikely to leak private information, allowing noise to be applied only to leaking inputs via the exponential mechanism. LUCID achieves tight iDP-DB bounds by combining MILP encodings, hyper-networks to summarize many related networks, and a novel branch-and-bound procedure with matching dependencies and relaxations to control complexity. Experimental results across four datasets and multiple architectures show that LUCID attains 0-iDP with about 1.4% accuracy loss (and 1.2% for 0<iDP), significantly outperforming DP-SGD and ALIBI baselines which incur around 12.7% loss. The approach is automatic, scalable, and adaptable to varying privacy budgets, offering a practical privacy-preserving alternative for label-only access scenarios.

Abstract

Neural networks are susceptible to privacy attacks that can extract private information of the training set. To cope, several training algorithms guarantee differential privacy (DP) by adding noise to their computation. However, DP requires to add noise considering every possible training set. This leads to a significant decrease in the network's accuracy. Individual DP (iDP) restricts DP to a given training set. We observe that some inputs deterministically satisfy iDP without any noise. By identifying them, we can provide iDP label-only access to the network with a minor decrease to its accuracy. However, identifying the inputs that satisfy iDP without any noise is highly challenging. Our key idea is to compute the iDP deterministic bound (iDP-DB), which overapproximates the set of inputs that do not satisfy iDP, and add noise only to their predicted labels. To compute the tightest iDP-DB, which enables to guard the label-only access with minimal accuracy decrease, we propose LUCID, which leverages several formal verification techniques. First, it encodes the problem as a mixed-integer linear program, defined over a network and over every network trained identically but without a unique data point. Second, it abstracts a set of networks using a hyper-network. Third, it eliminates the overapproximation error via a novel branch-and-bound technique. Fourth, it bounds the differences of matching neurons in the network and the hyper-network and employs linear relaxation if they are small. We show that LUCID can provide classifiers with a perfect individuals' privacy guarantee (0-iDP) -- which is infeasible for DP training algorithms -- with an accuracy decrease of 1.4%. For more relaxed $\varepsilon$-iDP guarantees, LUCID has an accuracy decrease of 1.2%. In contrast, existing DP training algorithms reduce the accuracy by 12.7%.

Figures (6)

• Figure 1: (a) Illustration of the subspaces of inputs with confidence over or below the iDP-DB on a 2D synthetic training set comprising 7,000 data points. (b) An overview of LUCID: given a dataset $D$, a training algorithm $\mathcal{T}$, and a classifier $N$, it computes the iDP-DB of every label (in this example, iDP-DB$_{c_0}=0.05$, iDP-DB$_{c_1}=0.01$) and returns iDP label-only access to $N$. Given an input $x$, this access computes $N(x)$ to identify the predicted label $c$ and the confidence $\mathcal{C}$ in $c$. If $\mathcal{C}>$ iDP-DB$_c$, it returns $c$; otherwise, it employs a noise mechanism. In this example, if a user submits to the iDP label-only access $x$ where $N(x) = (0.6,0.4)$, then $c=c_0$ and $\mathcal{C}=0.6-0.4>0.05$ and thus $c_0$ is returned. If $N(x) = (0.52,0.48)$, then $c=c_0$ and $\mathcal{C}=0.52-0.48\leq 0.05$ and thus a noise mechanism is used to select a class $\widetilde{c}\in\{c_0,c_1\}$ given a probability distribution $(P_0,P_1)$.
• Figure 2: The conceptually extended decision boundaries as a function of the input $(x_1,x_2)\in[0,1]^2$, for different confidences, including the iDP-DB.
• Figure 3: An example of a hyper-network and our branch-and-bound.
• Figure 4: An example of the matching dependencies and relax-if-similar.
• Figure 5: LUCID's component for computing the iDP-DB.

Theorems & Definitions (13)

• definition 1: Differential Privacy
• definition 2: Global Sensitivity
• definition 3: Individual Differential Privacy
• definition 4: Local Sensitivity
• definition 5: Problem Definition: Minimal Noise iDP Label-Only Queries
• lemma 1
• definition 6: Individual Differential Privacy Deterministic Bound (iDP-DB)
• definition 7: A Hyper-Network
• theorem 1
• lemma 1