Privacy-Preserving Black-Box Optimization (PBBO): Theory and the Model-Based Algorithm DFOp
Pengcheng Xie
TL;DR
PBBO tackles optimizing an objective observed only after iteration-dependent transformations F_k, formalized as minimize F(x) with F_k(x) = T_k(F(x)). The authors introduce DFOp, a model-based derivative-free solver that updates quadratic models via a least-Frobenius-norm criterion adapted to changing F_k and prove convergence for transformed objectives. They design two optimality-preserving differential-privacy mechanisms—an additive Laplace mechanism and a mixed multiplicative/additive mechanism—to encrypt the private component h in F = f + h, with explicit privacy budgets. Empirical results show that DFOp outperforms competing derivative-free solvers on encrypted/private PBBO tasks, closely matching noise-free baselines and demonstrating practical impact for privacy-aware optimization.
Abstract
This paper focuses on solving unconstrained privacy-preserving black-box optimization (PBBO), its corresponding least Frobenius norm updating of quadratic models, and the differentially privacy mechanisms for PBBO. Optimization problems with transformed/encrypted objective functions aim to minimize F(x), which is encrypted/transformed/encrypted to F_k(x) as the output at the k-th iteration. A new derivative-free solver named DFOp, with its implementation, is proposed in this paper, which has a new updating formula for the quadratic model functions. The convergence of DFOp for solving problems with transformed/encrypted objective functions is given. Other analyses, including the new model updating formula and the analysis of the transformation's impact to model functions are presented. We propose two differentially private noise-adding mechanisms for privacy-preserving black-box optimization. Numerical results show that DFOp performs better than compared algorithms. To the best of our knowledge, DFOp is the first derivative-free solver that can solve black-box optimization problems with step-encryption and privacy-preserving black-box problems exactly, which also tries to answer the open question about the combination of derivative-free optimization and privacy.