Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks

Alka Luqman, Yeow Wei Liang Brandon, Anupam Chattopadhyay

TL;DR

This work tackles the problem of optimizing knowledge transfer in dynamic federated learning by systematically comparing data-exchange strategies (raw data and synthetic data) with model-update exchanges, including foundations-model-based updates, under non-IID data and ad-hoc network conditions. It introduces a Peer-to-Peer FL framework (PeerFL) and an adaptive hybrid transfer algorithm that balances compute and communication costs through an action-reward mechanism, where $Reward(Action) = E[AccuracyIncrease|Action]$ and costs are captured by $C^{comm}$. Experimental results on CIFAR-10 with non-IID partitions and on KITTI object detection demonstrate that data-sharing approaches can achieve faster convergence at the expense of higher bandwidth, while model-sharing provides stability with fewer rounds; synthetic data can match raw data performance in several skewed scenarios, and pre-trained foundational models show diminishing returns in some regimes. The findings underscore the value of adaptive, hybrid strategies for real-world dynamic networks and suggest privacy-conscious extensions such as differential privacy and secure multi-party computation to further strengthen robustness and applicability.

Abstract

The promise and proliferation of large-scale dynamic federated learning gives rise to a prominent open question - is it prudent to share data or model across nodes, if efficiency of transmission and fast knowledge transfer are the prime objectives. This work investigates exactly that. Specifically, we study the choices of exchanging raw data, synthetic data, or (partial) model updates among devices. The implications of these strategies in the context of foundational models are also examined in detail. Accordingly, we obtain key insights about optimal data and model exchange mechanisms considering various environments with different data distributions and dynamic device and network connections. Across various scenarios that we considered, time-limited knowledge transfer efficiency can differ by up to 9.08\%, thus highlighting the importance of this work.

Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks

TL;DR

This work tackles the problem of optimizing knowledge transfer in dynamic federated learning by systematically comparing data-exchange strategies (raw data and synthetic data) with model-update exchanges, including foundations-model-based updates, under non-IID data and ad-hoc network conditions. It introduces a Peer-to-Peer FL framework (PeerFL) and an adaptive hybrid transfer algorithm that balances compute and communication costs through an action-reward mechanism, where and costs are captured by . Experimental results on CIFAR-10 with non-IID partitions and on KITTI object detection demonstrate that data-sharing approaches can achieve faster convergence at the expense of higher bandwidth, while model-sharing provides stability with fewer rounds; synthetic data can match raw data performance in several skewed scenarios, and pre-trained foundational models show diminishing returns in some regimes. The findings underscore the value of adaptive, hybrid strategies for real-world dynamic networks and suggest privacy-conscious extensions such as differential privacy and secure multi-party computation to further strengthen robustness and applicability.

Abstract

The promise and proliferation of large-scale dynamic federated learning gives rise to a prominent open question - is it prudent to share data or model across nodes, if efficiency of transmission and fast knowledge transfer are the prime objectives. This work investigates exactly that. Specifically, we study the choices of exchanging raw data, synthetic data, or (partial) model updates among devices. The implications of these strategies in the context of foundational models are also examined in detail. Accordingly, we obtain key insights about optimal data and model exchange mechanisms considering various environments with different data distributions and dynamic device and network connections. Across various scenarios that we considered, time-limited knowledge transfer efficiency can differ by up to 9.08\%, thus highlighting the importance of this work.
Paper Structure (9 sections, 3 equations, 3 figures, 2 tables, 3 algorithms)

This paper contains 9 sections, 3 equations, 3 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. Dataset & Preprocessing
  5. Federated Learning Framework
  6. Data & Model Exchange Strategies
  7. Metrics
  8. Experimental Studies
  9. Conclusion

Figures (3)

  • Figure 1: Workflow depicting Dataset Handler which utilizes an array of shuffling, sharding and image transformations to systematically generate realistic client datasets. Each client trains its own generator model to prevent data leakage while the communication handler dynamically adopts between data-sharing strategies and model-sharing strategies.
  • Figure 2: Synthetically generated samples of 5 images across classes of VAEs used in the Generator.
  • Figure 3: Results of 3 clients trained from scratch using model sharing strategy. Pathological non-i.i.d distribution seems to worsen all FedAvg, mostly due to missing classes being difficult to learn across multiple clients simultaneously. For client C however, it performs worse consistently across all distributions. Model sharing seems to contribute most when data samples are low and the data set may be more difficult, as seen from the larger improvement.