Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sketching Meets Differential Privacy: Fast Algorithm for Dynamic Kronecker Projection Maintenance

Zhao Song, Xin Yang, Yuanyuan Yang, Lichen Zhang

TL;DR

The paper tackles dynamic projection maintenance for Kronecker-structured products by maintaining the projection ${ m B}^ op({ m BB}^ op)^{-1}{ m B}$ where ${ m B}={ m A}(W ext{ or }W^{1/2})$-structured via low-rank updates to $W$ and online queries. It introduces a two-pronged approach: (i) a Kron-based projection maintenance data structure that leverages lazy eigenvalue updates, Woodbury identities, and batched sketching to achieve amortized subquadratic updates and efficient queries, and (ii) a differential privacy–driven robust set-query framework that defends against adaptive adversaries, reducing the required sketches to $ ilde{O}( ext{sqrt}(T))$ for single coordinates and to $ ilde{O}( ext{sqrt}(kT))$ for sets of coordinates. The main technical contributions are the Kronecker Product Projection Maintenance Data Structure (with Init/Update/Query guarantees and complexity under low-rank $W$ changes) and the Robust Set Query Data Structure (employing coordinate-wise embedding, PrivateMedian, and DP composition to ensure adaptivity resilience). Together, these results yield faster dynamic SDP-type computations and broader DP-based robustness for iterative projection maintenance tasks in Kronecker-structured settings. The work has potential impact on speeding convex optimization and SDP solvers that exploit Kronecker structure while providing strong guarantees under adaptive queries and updates.

Abstract

Projection maintenance is one of the core data structure tasks. Efficient data structures for projection maintenance have led to recent breakthroughs in many convex programming algorithms. In this work, we further extend this framework to the Kronecker product structure. Given a constraint matrix ${\sf A}$ and a positive semi-definite matrix $W\in \mathbb{R}^{n\times n}$ with a sparse eigenbasis, we consider the task of maintaining the projection in the form of ${\sf B}^\top({\sf B}{\sf B}^\top)^{-1}{\sf B}$, where ${\sf B}={\sf A}(W\otimes I)$ or ${\sf B}={\sf A}(W^{1/2}\otimes W^{1/2})$. At each iteration, the weight matrix $W$ receives a low rank change and we receive a new vector $h$. The goal is to maintain the projection matrix and answer the query ${\sf B}^\top({\sf B}{\sf B}^\top)^{-1}{\sf B}h$ with good approximation guarantees. We design a fast dynamic data structure for this task and it is robust against an adaptive adversary. Following the beautiful and pioneering work of [Beimel, Kaplan, Mansour, Nissim, Saranurak and Stemmer, STOC'22], we use tools from differential privacy to reduce the randomness required by the data structure and further improve the running time.

Sketching Meets Differential Privacy: Fast Algorithm for Dynamic Kronecker Projection Maintenance

TL;DR

The paper tackles dynamic projection maintenance for Kronecker-structured products by maintaining the projection where -structured via low-rank updates to and online queries. It introduces a two-pronged approach: (i) a Kron-based projection maintenance data structure that leverages lazy eigenvalue updates, Woodbury identities, and batched sketching to achieve amortized subquadratic updates and efficient queries, and (ii) a differential privacy–driven robust set-query framework that defends against adaptive adversaries, reducing the required sketches to for single coordinates and to for sets of coordinates. The main technical contributions are the Kronecker Product Projection Maintenance Data Structure (with Init/Update/Query guarantees and complexity under low-rank changes) and the Robust Set Query Data Structure (employing coordinate-wise embedding, PrivateMedian, and DP composition to ensure adaptivity resilience). Together, these results yield faster dynamic SDP-type computations and broader DP-based robustness for iterative projection maintenance tasks in Kronecker-structured settings. The work has potential impact on speeding convex optimization and SDP solvers that exploit Kronecker structure while providing strong guarantees under adaptive queries and updates.

Abstract

Projection maintenance is one of the core data structure tasks. Efficient data structures for projection maintenance have led to recent breakthroughs in many convex programming algorithms. In this work, we further extend this framework to the Kronecker product structure. Given a constraint matrix and a positive semi-definite matrix with a sparse eigenbasis, we consider the task of maintaining the projection in the form of , where or . At each iteration, the weight matrix receives a low rank change and we receive a new vector . The goal is to maintain the projection matrix and answer the query with good approximation guarantees. We design a fast dynamic data structure for this task and it is robust against an adaptive adversary. Following the beautiful and pioneering work of [Beimel, Kaplan, Mansour, Nissim, Saranurak and Stemmer, STOC'22], we use tools from differential privacy to reduce the randomness required by the data structure and further improve the running time.
Paper Structure (48 sections, 39 theorems, 125 equations, 4 figures, 1 table, 4 algorithms)

This paper contains 48 sections, 39 theorems, 125 equations, 4 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Roadmap.
  3. Related Work
  4. Sketching.
  5. Differential Privacy.
  6. Projection Maintenance Data Structure.
  7. Preliminaries & Problem Formulation
  8. Notations and Basic Definitions.
  9. Problem Formulation
  10. Technical Overview
  11. Kronecker Product Projection Maintenance Data Structure
  12. Set Query Data Structure
  13. Problem Definition
  14. Robust Set Query Data Structure
  15. Roadmap.
  16. ...and 33 more sections

Key Result

Theorem 1.2

Let ${\sf A}\in \mathbb{R}^{m\times n^2}$ and ${\sf B}={\sf A}(W\otimes I)$ or ${\sf B}={\sf A}(W^{1/2}\otimes W^{1/2})$. Let ${\sf R}\in \mathbb{R}^{sb\times n^2}$ be a batch of $s$ sketching matrices, each with dimension $b$. Given a sequence of online matrices $W^{(1)},\ldots,W^{(T)}\in \mathbb{R Here, $a\in (0,1)$ is a parameter that can be chosen and $f(a,c)\in [4,5)$ is a function defined as

Figures (4)

  • Figure 1: Online matrix vector multiplication (Definition \ref{['def:omv']}).
  • Figure 2: Online projection matrix vector multiplication (Definition \ref{['def:opmv']}). Usually, we say $\sqrt{W} A (A^\top W A)^{-1} A^\top \sqrt{W}$ is a projection matrix. We say $(A^\top W A)^{-1} A^\top \sqrt{W}$ is a projection matrix without left arm. We say $A (A^\top W A)^{-1} A^\top \sqrt{W}$ is a projection matrix without left hand. Technically, we call $\sqrt{W} A$ arm, and call $\sqrt{W}$ hand.
  • Figure 3: Online Kronecker matrix vector multiplication (Definition \ref{['def:okpmv']}), where $B_i = A_i W$, and the projection matrix is defined as $\mathsf{B}^\top (\mathsf{B} \mathsf{B}^\top)^{-1} \mathsf{B} = (W^\top \otimes I) \mathsf{A}^\top ( \mathsf{A} (W^2 \otimes I) \mathsf{A}^\top )^{-1} \mathsf{A} (W \otimes I)$.
  • Figure 4: Online Kronecker matrix vector multiplication (Definition \ref{['def:okpmv']}), where $B_i = W^{1/2}A_i W^{1/2}$, and the projection matrix is defined as $\mathsf{B}^\top (\mathsf{B} \mathsf{B}^\top)^{-1} \mathsf{B} = (W^{1/2} \otimes W^{1/2})^\top \mathsf{A}^\top ( \mathsf{A} (W \otimes W) \mathsf{A}^\top )^{-1} \mathsf{A} (W^{1/2} \otimes W^{1/2} )$.

Theorems & Definitions (93)

  • Definition 1.1: Time complexity for preprocessing, update and query
  • Theorem 1.2
  • Definition 3.1: Online Matrix Vector Multiplication (OMV), hkns15lw17ckl18
  • Definition 3.2: Online Projection Matrix Vector Multiplication (OPMV), cls19
  • Definition 3.3: Online Kronecker Projection Matrix Vector Multiplication(OKPMV)
  • Remark 3.4
  • Definition 5.1: cls19
  • Theorem 5.2: Kronecker Product Projection Maintenance. Informal version of Theorem \ref{['thm:app:kronecker_maintain']}
  • Lemma 5.3
  • Definition 6.1: Set Query
  • ...and 83 more