Adaptive Power Iteration Method for Differentially Private PCA
Ta Duy Nguyem, Alina Ene, Huy Le Nguyen
TL;DR
The paper proposes a private power iteration method for DP PCA that adapts to input coherence to achieve beyond-worst-case utility. By filtering datapoints adaptively per iteration using the Sparse Vector Technique and adding Gaussian noise to updates, the algorithm maintains row-level DP while achieving accurate recovery of the top singular vector under reasonable spectral-gap and coherence conditions. In the Gaussian/i.i.d. data setting, the method attains favorable sample complexity and coherence bounds, improving over worst-case DP PCA guarantees in low-coherence regimes. The work thus provides a practical, structure-exploiting approach to private PCA with per-row privacy, relevant to privacy-sensitive data pipelines in high-dimensional settings.
Abstract
We study $(ε,δ)$-differentially private algorithms for the problem of approximately computing the top singular vector of a matrix $A\in\mathbb{R}^{n\times d}$ where each row of $A$ is a datapoint in $\mathbb{R}^{d}$. In our privacy model, neighboring inputs differ by one single row/datapoint. We study the private variant of the power iteration method, which is widely adopted in practice. Our algorithm is based on a filtering technique which adapts to the coherence parameter of the input matrix. This technique provides a utility that goes beyond the worst-case guarantees for matrices with low coherence parameter. Our work departs from and complements the work by Hardt-Roth (STOC 2013) which designed a private power iteration method for the privacy model where neighboring inputs differ in one single entry by at most 1.