Multifold Confidence Intervals in Collaborative Mean Estimation (ColME) Using Sample Statistics

TL;DR

This paper studies the problem of personalized online mean estimation in heterogeneous environments, where each agent observes data from its own sigma-sub-Gaussian distribution and designs a unified procedure for constructing multifold CI based jointly on the sample mean, sample variance, and sample kurtosis.

Abstract

The rapid growth of digital devices and IoT has intensified the demand for collaborative learning. Since these devices generate sensitive and high-dimensional data, centralized transmission is often impractical, while local learning suffers from slow convergence. Collaborative approaches can alleviate these issues by allowing agents to use information from one another to improve estimation. Each agent faces a personalized learning problem, and collaboration is beneficial among agents whose data are generated from the same distributions. This paper studies the problem of personalized online mean estimation in heterogeneous environments, where each agent observes data from its own sigma-sub-Gaussian distribution. Collaborative algorithms enable agents to identify similarity classes in real time and exploit information from agents belonging to the the same class to improve convergence and accuracy. The work builds on existing approaches: the collaborative mean estimation (colME) and its graph-based extensions (C-colME and B-colME), which improve scalability and robustness. Since the variance estimation plays a crucial role in the above mentioned algorithms, a method for accurate, local and real-time estimation of variance is proposed. Estimation of sample kurtosis is also incorporated. We derive the CI estimators for the sample standard deviation and sample kurtosis. These results are combined with sample colME methods to design a unified procedure for constructing multifold CI based jointly on the sample mean, sample variance, and sample kurtosis. This framework enables colME in challenging scenarios, such as when classes share similar means but differ in variances, or when both means and variances are alike while the underlying distributions diverge in higher-order characteristics.

Figures (20)

• Figure 1: Illustration of Gaussian data, $x_a(t)$, $x_b(t)$, of two agents, $a$ and $b$, from different similarity classes (shown in red and blue dots) for $t \in [0,2000]$ with $\mu_a=0.1$, $\mu_b=0.9$, $\Delta_{ab}=0.8$, and $\sigma=2$.
• Figure 2: Illustration of local means, $\bar{x}_{aa}(t)$, of data $x_a(t)$ with 2 similarity classes and $N=200$ agents (shown in red and blue), with $\mu_a=0.1$, $\mu_b=0.9$, $\Delta_{ab}=|\mu_a-\mu_b|=0.8$, for $t \in [0,2000]$. Data $x_a(t)$ are Gaussian distributed with the standard deviation $\sigma=2$.
• Figure 3: Random regular graph with $A=100$ agents/nodes and $C=2$ classes, arbitrary connected, at $t=0$, with $r=4$ links using a random regular graph $\mathcal{G}(100, 4)$ .
• Figure 4: Histogram of the estimated sample standard deviation at different time instants $t$, converging to the derived theoretical distributions (dashed green and red line) as the number of data increases with $t$. High accuracy and agreement can be seen for $t=500$ and $t=2000$.
• Figure 5: B-colME on data with the same mean, $N = 1000$, $\delta=0.01$. Total MSE of the estimation averaged over all agents at time instants $t$ in 10 realizations. In top panel: Local estimate (green dashed line); Proposed method (blue line) with corresponding bootstrap confidence intervals; Oracle solution (red dotted line) with corresponding bootstrap confidence intervals. In bottom panel the MSE values for various depths $d=0, 1,2, 3, 4$ are shown, with $d=0$ coinciding with local estimate and $d=4$ being close to oracle, while the others are in between in corresponding order. Theoretical lines are dashed.