Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Binned Group Algebra Factorization for Differentially Private Continual Counting

Monika Henzinger, Nikita P. Kalinin, Jalaj Upadhyay

TL;DR

This work tackles private prefix sums under continual differential privacy by leveraging a group-algebra-based matrix factorization $M = L \times R$ to achieve improved accuracy over trivial noise methods. The authors establish that the group-algebra coefficients $b_f(\omega^t)$ are real and derive a real-valued decomposition, enabling a structural representation of $M$ in terms of the all-ones matrix $E$ and a matrix $C$, which underpins both norm bounds and monotonicity properties. To address practical efficiency, they adapt a binning approach to non-square factorizations, proving that one can construct a binned pair $(\hat{L},\hat{R})$ such that MeanSE and MaxSE are preserved within a factor $(1+\zeta)$ while achieving per-row computation time $O_{\zeta}(\sqrt{n}(\log n)^{3/2})$ and memory $O_{\zeta}(\sqrt{n}(\log n)^{3/2})$, with an $O(n)$-time construction. This work bridges theoretical improvements in factorization accuracy with practical efficiency for large-scale private learning systems. It opens avenues for extending these techniques to other workload matrices and exploring whether even tighter logarithmic error bounds are attainable in the group-algebra setting.

Abstract

We study memory-efficient matrix factorization for differentially private counting under continual observation. While recent work by Henzinger and Upadhyay 2024 introduced a factorization method with reduced error based on group algebra, its practicality in streaming settings remains limited by computational constraints. We present new structural properties of the group algebra factorization, enabling the use of a binning technique from Andersson and Pagh (2024). By grouping similar values in rows, the binning method reduces memory usage and running time to $\tilde O(\sqrt{n})$, where $n$ is the length of the input stream, while maintaining a low error. Our work bridges the gap between theoretical improvements in factorization accuracy and practical efficiency in large-scale private learning systems.

Binned Group Algebra Factorization for Differentially Private Continual Counting

TL;DR

This work tackles private prefix sums under continual differential privacy by leveraging a group-algebra-based matrix factorization to achieve improved accuracy over trivial noise methods. The authors establish that the group-algebra coefficients are real and derive a real-valued decomposition, enabling a structural representation of in terms of the all-ones matrix and a matrix , which underpins both norm bounds and monotonicity properties. To address practical efficiency, they adapt a binning approach to non-square factorizations, proving that one can construct a binned pair such that MeanSE and MaxSE are preserved within a factor while achieving per-row computation time and memory , with an -time construction. This work bridges theoretical improvements in factorization accuracy with practical efficiency for large-scale private learning systems. It opens avenues for extending these techniques to other workload matrices and exploring whether even tighter logarithmic error bounds are attainable in the group-algebra setting.

Abstract

We study memory-efficient matrix factorization for differentially private counting under continual observation. While recent work by Henzinger and Upadhyay 2024 introduced a factorization method with reduced error based on group algebra, its practicality in streaming settings remains limited by computational constraints. We present new structural properties of the group algebra factorization, enabling the use of a binning technique from Andersson and Pagh (2024). By grouping similar values in rows, the binning method reduces memory usage and running time to , where is the length of the input stream, while maintaining a low error. Our work bridges the gap between theoretical improvements in factorization accuracy and practical efficiency in large-scale private learning systems.

Paper Structure

This paper contains 18 sections, 14 theorems, 108 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contribution
  3. Preliminaries
  4. Results
  5. Group Algebra Properties
  6. Sparsification via Binning
  7. Conclusion and Open Questions
  8. Conclusion and Open Questions
  9. Appendix
  10. Proof of Lemma \ref{['lem:b_f_omega_is_real']}
  11. Proof of Lemma \ref{['lem:b_f_decomposition']}
  12. Proof of Lemma \ref{['lem:b_f_decreasing']}
  13. Proof of Lemma \ref{['lem:b_f_bounds']}
  14. Proof of Lemma \ref{['lem:L_last_row']}
  15. Proof of Lemma \ref{['lem:b_f_matrix']}
  16. ...and 3 more sections

Key Result

Theorem 1

(Main Theorem) Given a group algebra factorization $M = L \times R$, where $L = (L_1, L_2) \in \mathbb{R}^{n\times 2n}$ and $R = (R_1, R_2)^T \in \mathbb{R}^{2n \times n}$, we can find a binned matrix $\hat{L} = (\hat{L}_1, \hat{L}_2)$ and a corresponding matrix $\hat{R} = (\hat{L}_1^{-1}L_1R_1, \ha Moreover, the product of the matrix $\hat{L}$ with a vector $z \in \mathbb{R}^{2n}$ can be computed

Figures (1)

  • Figure :

Theorems & Definitions (28)

  • Theorem 1
  • Remark : Difference from Related Works
  • Theorem 2: Theorem 1.1 from group_algebra
  • Definition 1: $(\eta, \mu)$-perturbation
  • Definition 2: Binned Matrix
  • Lemma 1: Factorization is real-valued
  • Lemma 2: Closed-form expression of factors
  • Lemma 3: Monotonicity Property
  • Lemma 4
  • Lemma 5
  • ...and 18 more