Constant matters: Fine-grained Complexity of Differentially Private Continual Observation

TL;DR

The paper presents a fine-grained, constant-aware analysis of differential privacy for counting under continual observation by leveraging a matrix mechanism with an explicit, lower-triangular factorization of the counting matrix M_count. It derives a closed-form cb-norm bound Psi(T) and uses this to obtain concrete, per-round additive error bounds for continual counting, histogram maintenance, and a range of graph and string-analytic tasks, all implemented via time-efficient L,R factorizations. The approach yields monotone, smooth error behavior and enables applications to synthetic graph construction, substring and episode counting, and non-interactive local learning, while also establishing lower bounds for data-independent mechanisms. Experimental results confirm that the proposed matrix mechanism achieves substantially tighter and smoother error profiles than the classic binary mechanism, with practical implications for privacy-preserving continual data analysis.

Abstract

We study fine-grained error bounds for differentially private algorithms for counting under continual observation. Our main insight is that the matrix mechanism when using lower-triangular matrices can be used in the continual observation model. More specifically, we give an explicit factorization for the counting matrix $M_\mathsf{count}$ and upper bound the error explicitly. We also give a fine-grained analysis, specifying the exact constant in the upper bound. Our analysis is based on upper and lower bounds of the {\em completely bounded norm} (cb-norm) of $M_\mathsf{count}$. Along the way, we improve the best-known bound of 28 years by Mathias (SIAM Journal on Matrix Analysis and Applications, 1993) on the cb-norm of $M_\mathsf{count}$ for a large range of the dimension of $M_\mathsf{count}$. Furthermore, we are the first to give concrete error bounds for various problems under continual observation such as binary counting, maintaining a histogram, releasing an approximately cut-preserving synthetic graph, many graph-based statistics, and substring and episode counting. Finally, we note that our result can be used to get a fine-grained error bound for non-interactive local learning {and the first lower bounds on the additive error for $(ε,δ)$-differentially-private counting under continual observation.} Subsequent to this work, Henzinger et al. (SODA2023) showed that our factorization also achieves fine-grained mean-squared error.

Figures (5)

• Figure 1: Additive $\ell_{\infty}$ error with $T=2^{16}, \epsilon = 0.8, \delta = 10^{-10}$.
• Figure 2: Difference between our upper bound and the explicitly computed Mathias lower bound.
• Figure 3: Comparison of our mechanism with binary mechanism for $T=2^{16}, \epsilon=0.5,\delta=10^{-10}$ and various sparsity level. The $x$-axis is the current time epoch, the $y$-axis gives the output of the algorithms at each time epoch.
• Figure 4: (Left) We give the Zipf’s law distribution that items are sampled from at each round of an event stream. (Right) the running estimate of the most frequent item using the binary mechanism and our mechanism instantiated with $\epsilon=0.1, \delta = 10^{-10}$.
• Figure :

Theorems & Definitions (34)

• Theorem 1: Upper bound on $\left\| M_\mathsf{count} \right\|_\mathsf{cb}$
• Theorem 2: Upper bound on differentially private continual counting
• Remark 1: Suboptimality of the binary mechanism with respect to the constant
• Theorem 3: Lower bound
• Definition 1
• Definition 2: Gaussian mechanism
• Definition 3: Accuracy
• Lemma 1: chen2005best
• proof : Proof of \ref{['thm:cb_bound']}
• Lemma 2