Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labels

TL;DR

CF-CL addresses unsupervised federated learning at the edge by enabling device-to-device push-pull exchanges of either explicit datapoints or implicit embeddings to align local latent spaces under non-i.i.d. distributions. It introduces a probabilistic, two-stage importance sampling framework and reserve data for explicit exchange, along with a matching embedding-based scheme and a regularized triplet loss to incorporate cross-device information for implicit exchange. The method yields faster convergence and better latent-space alignment than baselines while offering tradeoffs between communication efficiency and latency, depending on whether explicit or implicit information is used. Overall, CF-CL provides a practical, privacy-conscious, distributed approach to coordinated unsupervised learning across edge devices with limited central coordination.

Abstract

Federated learning (FL) is a popular solution for distributed machine learning (ML). While FL has traditionally been studied for supervised ML tasks, in many applications, it is impractical to assume availability of labeled data across devices. To this end, we develop Cooperative Federated unsupervised Contrastive Learning ({\tt CF-CL)} to facilitate FL across edge devices with unlabeled datasets. {\tt CF-CL} employs local device cooperation where either explicit (i.e., raw data) or implicit (i.e., embeddings) information is exchanged through device-to-device (D2D) communications to improve local diversity. Specifically, we introduce a \textit{smart information push-pull} methodology for data/embedding exchange tailored to FL settings with either soft or strict data privacy restrictions. Information sharing is conducted through a probabilistic importance sampling technique at receivers leveraging a carefully crafted reserve dataset provided by transmitters. In the implicit case, embedding exchange is further integrated into the local ML training at the devices via a regularization term incorporated into the contrastive loss, augmented with a dynamic contrastive margin to adjust the volume of latent space explored. Numerical evaluations demonstrate that {\tt CF-CL} leads to alignment of latent spaces learned across devices, results in faster and more efficient global model training, and is effective in extreme non-i.i.d. data distribution settings across devices.

Figures (11)

• Figure 1: CF-CL introduces smart push-pull information transfer to improve unsupervised FL based on importance information sampling.
• Figure 2: (a) A datapoint (anchor), its augmentation (positive) and a distinct datapoint (negative), are passed through models to obtain embeddings (b). Training maximizes distance between anchor and negative, while minimizing distance between anchor and positive (c).
• Figure 3: Centroids of remote clusters (green) that are closer to local clusters (red) have a higher chance of being similar to local data (Left), while centroids that are further away are less likely (Right).
• Figure 4: Distance between embeddings of pairwise combinations of labels. Information exchange results in dissimilar embeddings being further apart from each other.
• Figure 5: Training performance comparison of CF-CL against baselines over training iterations, for each dataset and information exchange regime. We see that CF-CL has superior performance in all cases, validating the benefit of its information sharing mechanism for local model alignment.