Fixed-Budget Differentially Private Best Arm Identification

TL;DR

This work addresses the problem of identifying the best arm in a linear bandit under a fixed budget while enforcing $\varepsilon$-differential privacy. The authors introduce DP-BAI, a policy built around the Max-Det design to minimize privacy-induced noise and a phase-based allocation that exploits the linear structure; private empirical means are formed by Laplace noise, with non-pulled arms inferred via a coordinate representation in a Max-Det basis. They prove that DP-BAI satisfies $\varepsilon$-DP and achieves an exponential decay of the error probability with the budget, with an upper bound $\mathbb{P}_v^{\Pi_{DP-BAI}}(\hat{I}_T \neq i^*(v)) \le \exp(-T'/(65\,M\,H))$, where $M=\Theta(\log d)$ and $T'=\Theta(T)$, and $H(v)=H_{BAI}(v)+H_{pri}(v)$. A corresponding minimax lower bound shows that the exponential rate is tight up to polylog factors, with the hardness term decomposing into a standard BAI component and a privacy component $H_{pri}(v)=(1/\varepsilon)\max_{2\le i\le(d^2\wedge K)} i/\Delta_{(i)}$. Numerical results corroborate the theoretical findings, showing DP-BAI outperforms privacy-aware baselines and approaches the non-private state of the art as privacy vanishes. The work thus advances understanding of private pure-exploration in fixed-budget linear bandits and suggests directions for broader privacy-preserving exploration in bandits.

Abstract

We study best arm identification (BAI) in linear bandits in the fixed-budget regime under differential privacy constraints, when the arm rewards are supported on the unit interval. Given a finite budget $T$ and a privacy parameter $\varepsilon>0$, the goal is to minimise the error probability in finding the arm with the largest mean after $T$ sampling rounds, subject to the constraint that the policy of the decision maker satisfies a certain {\em $\varepsilon$-differential privacy} ($\varepsilon$-DP) constraint. We construct a policy satisfying the $\varepsilon$-DP constraint (called {\sc DP-BAI}) by proposing the principle of {\em maximum absolute determinants}, and derive an upper bound on its error probability. Furthermore, we derive a minimax lower bound on the error probability, and demonstrate that the lower and the upper bounds decay exponentially in $T$, with exponents in the two bounds matching order-wise in (a) the sub-optimality gaps of the arms, (b) $\varepsilon$, and (c) the problem complexity that is expressible as the sum of two terms, one characterising the complexity of standard fixed-budget BAI (without privacy constraints), and the other accounting for the $\varepsilon$-DP constraint. Additionally, we present some auxiliary results that contribute to the derivation of the lower bound on the error probability. These results, we posit, may be of independent interest and could prove instrumental in proving lower bounds on error probabilities in several other bandit problems. Whereas prior works provide results for BAI in the fixed-budget regime without privacy constraints or in the fixed-confidence regime with privacy constraints, our work fills the gap in the literature by providing the results for BAI in the fixed-budget regime under the $\varepsilon$-DP constraint.

Figures (2)

• Figure 1: Comparison of DP-BAI to Baseline, OD-LinBAI and DP-OD for different values of $\varepsilon$. Note that $\varepsilon$ is not applicable to OD-LinBAI.
• Figure 2: Comparison of DP-BAI to DP-OD for different values of $\frac{x\lor y}{x\land y}$ in the two-armed bandit instance introduced in Appendix \ref{['sec:DPODanalysis']}.

Theorems & Definitions (58)

• Definition 2.1
• Remark 1
• Definition 3.1
• Remark 2
• Proposition 4.1
• Theorem 4.2
• Lemma 4.3
• Corollary 4.4
• Lemma 4.5
• Definition 4.6