Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Communication-Constrained Private Decentralized Online Personalized Mean Estimation

Yauhen Yakimenka, Hsuan-Yin Lin, Eirik Rosnes, Jörg Kliewer

TL;DR

This work tackles private, decentralized online mean estimation under communication constraints by integrating differential privacy with a consensus-based algorithm (Private-C-ColME). The authors prove that, under an oracle decision rule and certain connectivity/privacy conditions, collaboration yields faster convergence than fully local updates, achieving an $O(1/t)$ mean-squared error improvement that depends on the class sizes $n_a$ and DP noise $\sigma_{\mathrm{DP}}^2$. They establish convergence bounds and privacy guarantees, and validate the approach through numerical experiments over multi-class, graph-constrained networks, showing practical gains when DP noise is appropriately bounded. The proposed framework enables real-time, personalized mean estimation in online settings while preserving data privacy and respecting communication limitations, with decision rules (Bernstein-based or optimistic distance) to identify same-mean neighbors. Overall, the paper contributes a theoretically grounded, privacy-preserving, communication-aware decentralized scheme for faster online personalization. The results underscore the trade-offs between privacy, connectivity, and collaboration gains in distributed mean estimation tasks.

Abstract

We consider the problem of communication-constrained collaborative personalized mean estimation under a privacy constraint in an environment of several agents continuously receiving data according to arbitrary unknown agent-specific distributions. A consensus-based algorithm is studied under the framework of differential privacy in order to protect each agent's data. We give a theoretical convergence analysis of the proposed consensus-based algorithm for any bounded unknown distributions on the agents' data, showing that collaboration provides faster convergence than a fully local approach where agents do not share data, under an oracle decision rule and under some restrictions on the privacy level and the agents' connectivity, which illustrates the benefit of private collaboration in an online setting under a communication restriction on the agents. The theoretical faster-than-local convergence guarantee is backed up by several numerical results.

Communication-Constrained Private Decentralized Online Personalized Mean Estimation

TL;DR

This work tackles private, decentralized online mean estimation under communication constraints by integrating differential privacy with a consensus-based algorithm (Private-C-ColME). The authors prove that, under an oracle decision rule and certain connectivity/privacy conditions, collaboration yields faster convergence than fully local updates, achieving an mean-squared error improvement that depends on the class sizes and DP noise . They establish convergence bounds and privacy guarantees, and validate the approach through numerical experiments over multi-class, graph-constrained networks, showing practical gains when DP noise is appropriately bounded. The proposed framework enables real-time, personalized mean estimation in online settings while preserving data privacy and respecting communication limitations, with decision rules (Bernstein-based or optimistic distance) to identify same-mean neighbors. Overall, the paper contributes a theoretically grounded, privacy-preserving, communication-aware decentralized scheme for faster online personalization. The results underscore the trade-offs between privacy, connectivity, and collaboration gains in distributed mean estimation tasks.

Abstract

We consider the problem of communication-constrained collaborative personalized mean estimation under a privacy constraint in an environment of several agents continuously receiving data according to arbitrary unknown agent-specific distributions. A consensus-based algorithm is studied under the framework of differential privacy in order to protect each agent's data. We give a theoretical convergence analysis of the proposed consensus-based algorithm for any bounded unknown distributions on the agents' data, showing that collaboration provides faster convergence than a fully local approach where agents do not share data, under an oracle decision rule and under some restrictions on the privacy level and the agents' connectivity, which illustrates the benefit of private collaboration in an online setting under a communication restriction on the agents. The theoretical faster-than-local convergence guarantee is backed up by several numerical results.

Paper Structure

This paper contains 17 sections, 8 theorems, 21 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Differential Privacy
  5. Bernstein's Condition
  6. Hypothesis Testing
  7. Model and Algorithm
  8. System Model (Problem Formulation)
  9. Private-C-ColME Algorithm
  10. Decision Rule
  11. Bernstein Rule
  12. Optimistic Distance
  13. Privacy and Convergence
  14. Privacy
  15. Convergence
  16. ...and 2 more sections

Key Result

Lemma 1

Let $(x_1,\dotsc,x_n) \in \mathcal{X}^n$ where $\mathcal{X} = [\mu-L,\mu+L]$ for some finite values $\mu$ and $L$. Then, the noise-corrupted sample mean $(x_1+\cdots+x_n)/n + Z/n$, where $Z \sim \mathcal{L}\left(0,\sigma_{\mathrm{DP}}/\sqrt{2} \right)$ and $\sigma_{\mathrm{DP}}^2 \triangleq 8L^2/\ep

Figures (1)

  • Figure 1: Comparing the average MSE of Private-C-ColME for different privacy levels $\epsilon$ and three different decision rules: oracle (solid curves), Bernstein hypothesis testing (dotted), and optimistic distance (dashdotted). There are $M=200$ agents forming three classes with $r=20$ (left-most plot) and $r=5$ (right-most plot). The curves are for uniform data with standard deviation $\sigma=1/2$ and $L = \sigma \sqrt{3}$. The results are based on $4000$ simulation runs.

Theorems & Definitions (10)

  • Definition 1
  • Lemma 1
  • Definition 2: Wainwright19_1
  • Lemma 2
  • Lemma 3
  • Lemma 4: Wainwright19_1
  • Theorem 1
  • Corollary 1
  • Proposition 1: Local
  • Proposition 2: Ideal