Privacy-Preserving Distributed Estimation with Limited Data Rate

TL;DR

The paper tackles privacy leakage in distributed estimation under limited data rate by introducing a binary-valued quantizer-based algorithm. It leverages Fisher information as a privacy metric to show that, with 1-bit communications per link and time-varying privacy noises, privacy improves dynamically and can decay to zero at a polynomial rate, while still ensuring almost-sure convergence to the true parameter. A co-design framework for privacy noises and step-sizes guarantees convergence even with heavy-tailed or increasing privacy noises, and a quantified trade-off reveals that stronger privacy slows convergence. The approach achieves strong privacy with minimal communication overhead and is validated through numerical simulations and a health-data experiment, highlighting practical impact for privacy-preserving, low-bandwidth networked estimation.

Abstract

This paper focuses on the privacy-preserving distributed estimation problem with a limited data rate, where the observations are the sensitive information. Specifically, a binary-valued quantizer-based privacy-preserving distributed estimation algorithm is developed, which improves the algorithm's privacy-preserving capability and simultaneously reduces the communication costs. The algorithm's privacy-preserving capability, measured by the Fisher information matrix, is dynamically enhanced over time. Notably, the Fisher information matrix of the output signals with respect to the sensitive information converges to zero at a polynomial rate, and the improvement in privacy brought by the quantizers is quantitatively characterized as a multiplicative effect. Regarding the communication costs, each sensor transmits only 1 bit of information to its neighbours at each time step. Additionally, the assumption on the negligible quantization error for real-valued messages is not required. While achieving the requirements of privacy preservation and reducing communication costs, the algorithm ensures that its estimates converge almost surely to the true value of the unknown parameter by establishing a co-design guideline for the time-varying privacy noises and step-sizes. A polynomial almost sure convergence rate is obtained, and then the trade-off between privacy and convergence rate is established. Numerical examples demonstrate the main results.

Figures (6)

• Figure 1: Communication graphs
• Figure 2: The trajectories of $\ln\left( \frac{1}{100N} \sum_{i=1}^{N} \sum_{\varsigma = 1}^{100} \lVert \tilde{\uptheta}_{i,k}^{\varsigma} \rVert^2 \right)$
• Figure 3: The upper boundaries of the non-zero elements in $\mathbb{E} \mathcal{I}_{\mathtt{S}}(\mathtt{y}_{i,k})$ for the sensors $1$ and $2$
• Figure 4: The trade-off between privacy and convergence rate
• Figure 5: The trajectories of $\ln\left(\frac{1}{100nN} \sum_{i=1}^{N} \sum_{\varsigma = 1}^{100} \lVert \tilde{\uptheta}_{i,k}^{\varsigma} \rVert^2 \right)$

Theorems & Definitions (75)

• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Remark 5
• Definition 1: Fisher information, zamir1998Fisher
• Proposition 1: Cramér-Rao lower bound, zamir1998Fisher
• Remark 6
• Definition 2
• Remark 7