Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentially Private Online Bayesian Estimation With Adaptive Truncation

Sinan Yıldırım

TL;DR

This work tackles online differentially private estimation of a static population parameter $\theta$ from sequentially collected privatized data $Y_t$ where $X_t \sim P_\theta$. It combines online Sequential Monte Carlo for Bayesian updating with an adaptive truncation of queries, selecting truncation intervals via a Thompson-sampling–style scheme guided by Fisher information of the noisy observations. The main contributions are a principled online DP estimation framework with adaptive queries and a practical interval-selection approach that leverages location-scale structure to precompute optimal truncation bounds, demonstrated through numerical experiments on Gaussian data. The results show that adaptive truncation yields more accurate estimates under a fixed privacy budget, enabling more effective privacy-preserving streaming inference in settings like continuous monitoring and online data collection.

Abstract

We propose a novel online and adaptive truncation method for differentially private Bayesian online estimation of a static parameter regarding a population. We assume that sensitive information from individuals is collected sequentially and the inferential aim is to estimate, on-the-fly, a static parameter regarding the population to which those individuals belong. We propose sequential Monte Carlo to perform online Bayesian estimation. When individuals provide sensitive information in response to a query, it is necessary to perturb it with privacy-preserving noise to ensure the privacy of those individuals. The amount of perturbation is proportional to the sensitivity of the query, which is determined usually by the range of the queried information. The truncation technique we propose adapts to the previously collected observations to adjust the query range for the next individual. The idea is that, based on previous observations, we can carefully arrange the interval into which the next individual's information is to be truncated before being perturbed with privacy-preserving noise. In this way, we aim to design predictive queries with small sensitivity, hence small privacy-preserving noise, enabling more accurate estimation while maintaining the same level of privacy. To decide on the location and the width of the interval, we use an exploration-exploitation approach a la Thompson sampling with an objective function based on the Fisher information of the generated observation. We show the merits of our methodology with numerical examples.

Differentially Private Online Bayesian Estimation With Adaptive Truncation

TL;DR

This work tackles online differentially private estimation of a static population parameter from sequentially collected privatized data where . It combines online Sequential Monte Carlo for Bayesian updating with an adaptive truncation of queries, selecting truncation intervals via a Thompson-sampling–style scheme guided by Fisher information of the noisy observations. The main contributions are a principled online DP estimation framework with adaptive queries and a practical interval-selection approach that leverages location-scale structure to precompute optimal truncation bounds, demonstrated through numerical experiments on Gaussian data. The results show that adaptive truncation yields more accurate estimates under a fixed privacy budget, enabling more effective privacy-preserving streaming inference in settings like continuous monitoring and online data collection.

Abstract

We propose a novel online and adaptive truncation method for differentially private Bayesian online estimation of a static parameter regarding a population. We assume that sensitive information from individuals is collected sequentially and the inferential aim is to estimate, on-the-fly, a static parameter regarding the population to which those individuals belong. We propose sequential Monte Carlo to perform online Bayesian estimation. When individuals provide sensitive information in response to a query, it is necessary to perturb it with privacy-preserving noise to ensure the privacy of those individuals. The amount of perturbation is proportional to the sensitivity of the query, which is determined usually by the range of the queried information. The truncation technique we propose adapts to the previously collected observations to adjust the query range for the next individual. The idea is that, based on previous observations, we can carefully arrange the interval into which the next individual's information is to be truncated before being perturbed with privacy-preserving noise. In this way, we aim to design predictive queries with small sensitivity, hence small privacy-preserving noise, enabling more accurate estimation while maintaining the same level of privacy. To decide on the location and the width of the interval, we use an exploration-exploitation approach a la Thompson sampling with an objective function based on the Fisher information of the generated observation. We show the merits of our methodology with numerical examples.
Paper Structure (13 sections, 6 theorems, 31 equations, 3 figures, 4 algorithms)

This paper contains 13 sections, 6 theorems, 31 equations, 3 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Differential Privacy
  3. Differentially private parameter estimation with adaptive queries
  4. Sequential Monte Carlo for online Bayesian estimation
  5. Adaptive truncation for the transformation
  6. Exploration-exploitation for interval selection
  7. Thompson sampling.
  8. Location and scale parameters and truncation
  9. Numerical results
  10. Conclusion
  11. Proof of Proposition \ref{['prop: DP of Alg 1']}
  12. Proof of Theorem \ref{['thm: order F']}
  13. Supplementary algorithms

Key Result

Theorem 1

Define functions $A_{1}: \mathscr{X} \times \mathcal{E}_{1} \mapsto \mathcal{Y}_{1}$ and $A_{2}: \mathcal{Y}_{1} \times \mathcal{E}_{2} \mapsto \mathcal{Y}_{2}$; and probability distributions $\mu_{1}$, $\mu_{2}$ on $\mathcal{E}_{1}$, $\mathcal{E}_{2}$, respectively. Furthermore, let $A: \mathscr{X}

Figures (3)

  • Figure 1: The entire course of one run of the SMC method with adaptive truncation
  • Figure 2: The particle distribution of SMC (every 20th time-step shown) (blue points) and the estimate of the posterior means (red line) versus time. Black lines indicate the true values. In plots for $\mu$, truncation points are also shown.
  • Figure 3: Box-plots of the posterior means, obtained from 30 runs

Theorems & Definitions (13)

  • Definition 1: Differential privacy (DP)
  • Theorem 1: Post-processing
  • Theorem 2: Laplace mechanism
  • Example 1
  • Proposition 1
  • Definition 2
  • Theorem 3
  • proof
  • Lemma 1
  • proof
  • ...and 3 more