Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimax and Adaptive Covariance Matrix Estimation under Differential Privacy

T. Tony Cai, Yicheng Li

Abstract

The covariance matrix plays a fundamental role in the analysis of high-dimensional data. This paper studies minimax and adaptive estimation of high-dimensional bandable covariance matrices under differential privacy constraints. We propose a novel differentially private blockwise tridiagonal estimator that achieves minimax-optimal convergence rates under both the operator norm and the Frobenius norm. In contrast to the non-private setting, the privacy-induced error exhibits a polynomial dependence on the ambient dimension, revealing a substantial additional cost of privacy. To establish optimality, we develop a new differentially private van Trees inequality and construct carefully designed prior distributions to obtain matching minimax lower bounds. The proposed private van Trees inequality applies more broadly to general private estimation problems and is of independent interest. We further introduce an adaptive estimator that attains the optimal rate up to a logarithmic factor without prior knowledge of the decay parameter, based on a novel hierarchical tridiagonal approach. Numerical experiments corroborate the theoretical results and illustrate the fundamental privacy-accuracy trade-off.

Minimax and Adaptive Covariance Matrix Estimation under Differential Privacy

Abstract

The covariance matrix plays a fundamental role in the analysis of high-dimensional data. This paper studies minimax and adaptive estimation of high-dimensional bandable covariance matrices under differential privacy constraints. We propose a novel differentially private blockwise tridiagonal estimator that achieves minimax-optimal convergence rates under both the operator norm and the Frobenius norm. In contrast to the non-private setting, the privacy-induced error exhibits a polynomial dependence on the ambient dimension, revealing a substantial additional cost of privacy. To establish optimality, we develop a new differentially private van Trees inequality and construct carefully designed prior distributions to obtain matching minimax lower bounds. The proposed private van Trees inequality applies more broadly to general private estimation problems and is of independent interest. We further introduce an adaptive estimator that attains the optimal rate up to a logarithmic factor without prior knowledge of the decay parameter, based on a novel hierarchical tridiagonal approach. Numerical experiments corroborate the theoretical results and illustrate the fundamental privacy-accuracy trade-off.
Paper Structure (55 sections, 40 theorems, 249 equations, 8 figures, 4 algorithms)

This paper contains 55 sections, 40 theorems, 249 equations, 8 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Main Contribution
  3. Related Work
  4. Organization
  5. Notations and Assumptions
  6. Covariance Estimation under DP
  7. Mechanism for DP
  8. Blockwise Tridiagonal Estimator
  9. Estimation Error under Operator Norm
  10. Estimation Error under Frobenius Norm
  11. Discussions
  12. Minimax Optimal Rates
  13. Block Size Parameter
  14. Dimension-Dependent Cost of Privacy
  15. Operator Norm vs. Frobenius Norm
  16. ...and 40 more sections

Key Result

Lemma 1.2

If $M$ is $(\epsilon, 0)$-DP, then $M$ is $(\epsilon^2/2)$-zCDP. Conversely, If $M$ is $\rho$-zCDP, then $M$ is $\left( \rho + 2 \sqrt{\rho \log (1/\delta)}, \delta \right)$-DP for every $\delta \in (0,1)$. In particular, for $\epsilon \in (0,1]$ and $\delta \in (0, e^{-1}]$, $\rho$-zCDP with $\rho

Figures (8)

  • Figure 1: Illustration of the blockwise tridiagonal estimator
  • Figure 2: Illustration of the adaptive blockwise tridiagonal estimator. Different colors represent different levels of blocks. The left figure: overall structure. The right figures: blocks at each level consist of blockwise tridiagonal matrices of increasing sizes. The red blocks are of size $k_0$ and the size doubles at each level.
  • Figure 3: Comparison of the true covariance matrix and the estimators.
  • Figure 4: Estimation errors of the DP blockwise tridiagonal estimator under different privacy budgets.
  • Figure 5: Convergence rates of the DP blockwise tridiagonal estimator and the adaptive estimator under two asymptotic regimes.
  • ...and 3 more figures

Theorems & Definitions (58)

  • Definition 1.1: $\rho$-zCDP
  • Lemma 1.2
  • Lemma 2.1: Gaussian Mechanism
  • Lemma 2.2: Composition
  • Proposition 2.3
  • Proposition 2.4
  • Theorem 2.5
  • Corollary 2.6
  • Theorem 2.7
  • Corollary 2.8
  • ...and 48 more