Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning

Seohyun Lee, Anindya Bijoy Das, Satyavrat Wagle, Christopher G. Brinton

TL;DR

The paper addresses the challenge of non-i.i.d. data in unsupervised federated learning by introducing a reinforcement-learning-based method to discover an optimal device-to-device data exchange graph. It combines PCA for feature preservation, K-means++ for measuring cross-device dissimilarity, and autoencoders for unsupervised learning, enabling selective data transfers that improve convergence speed and robustness to stragglers. The authors propose a decentralized RL framework where each device learns which neighbor to connect to, guided by a reward that balances data diversity and communication reliability, and they validate the approach on FashionMNIST and CIFAR-10 across multiple FL schemes (FedAvg, FedSGD, FedProx). The results demonstrate faster reconstruction loss convergence, more discriminative latent embeddings, and greater resilience to transmission failures and stragglers, suggesting practical applicability as a plug-and-play enhancement for unsupervised FL with D2D exchanges.

Abstract

One of the main challenges of decentralized machine learning paradigms such as Federated Learning (FL) is the presence of local non-i.i.d. datasets. Device-to-device transfers (D2D) between distributed devices has been shown to be an effective tool for dealing with this problem and robust to stragglers. In an unsupervised case, however, it is not obvious how data exchanges should take place due to the absence of labels. In this paper, we propose an approach to create an optimal graph for data transfer using Reinforcement Learning. The goal is to form links that will provide the most benefit considering the environment's constraints and improve convergence speed in an unsupervised FL environment. Numerical analysis shows the advantages in terms of convergence speed and straggler resilience of the proposed method to different available FL schemes and benchmark datasets.

Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning

TL;DR

The paper addresses the challenge of non-i.i.d. data in unsupervised federated learning by introducing a reinforcement-learning-based method to discover an optimal device-to-device data exchange graph. It combines PCA for feature preservation, K-means++ for measuring cross-device dissimilarity, and autoencoders for unsupervised learning, enabling selective data transfers that improve convergence speed and robustness to stragglers. The authors propose a decentralized RL framework where each device learns which neighbor to connect to, guided by a reward that balances data diversity and communication reliability, and they validate the approach on FashionMNIST and CIFAR-10 across multiple FL schemes (FedAvg, FedSGD, FedProx). The results demonstrate faster reconstruction loss convergence, more discriminative latent embeddings, and greater resilience to transmission failures and stragglers, suggesting practical applicability as a plug-and-play enhancement for unsupervised FL with D2D exchanges.

Abstract

One of the main challenges of decentralized machine learning paradigms such as Federated Learning (FL) is the presence of local non-i.i.d. datasets. Device-to-device transfers (D2D) between distributed devices has been shown to be an effective tool for dealing with this problem and robust to stragglers. In an unsupervised case, however, it is not obvious how data exchanges should take place due to the absence of labels. In this paper, we propose an approach to create an optimal graph for data transfer using Reinforcement Learning. The goal is to form links that will provide the most benefit considering the environment's constraints and improve convergence speed in an unsupervised FL environment. Numerical analysis shows the advantages in terms of convergence speed and straggler resilience of the proposed method to different available FL schemes and benchmark datasets.
Paper Structure (15 sections, 8 equations, 6 figures, 2 algorithms)

This paper contains 15 sections, 8 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Summary of Contributions
  3. System Model
  4. Federated Learning Setup
  5. Trust Between Devices
  6. D2D communication
  7. Optimal Graph Discovery
  8. Deciding the Best Link
  9. Data Transfers
  10. Proposed Methodology
  11. Distance and Graph Formation
  12. Data Exchange
  13. Learning Task
  14. Numerical Analysis
  15. Conclusion

Figures (6)

  • Figure 1: Environment for D2D-enabled FL for heterogeneous networks (a possible network may include phones, tablets or other devices). Each local device acts as its own agent in the RL formulation.
  • Figure 2: Process of updating the Q-table and selecting the optimal action for a given local device $c_i$
  • Figure 3: Dissimilarity between clients across 10 devices with FMNIST. Left is before D2D and right is after D2D, with average value of 6.24 and 5.61, respectively.
  • Figure 4: Probability of failure of links formed at RL and uniform. Our method significantly improves the probability of successful transmission compared to uniform across both datasets.
  • Figure 5: Reconstruction Loss and Linear Evaluation with (a) FedAvg, (b) FedSGD, and (c) FedProx. Our method significantly improves the loss of the global model across all three FL algorithms carried out on a FL setting with $N=30$ clients. Significant increase in accuracy values also indicates the latent embeddings of the different classes with our proposed method are more spread out, allowing for better classification.
  • ...and 1 more figures