Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedAvgen: Metadata for Model Aggregation In Communication Systems

Anthony Kiggundu, Dennis Krummacker, Hans D. Schotten

TL;DR

The multiplicity in profiles motivates the conceptual assessment of a metaheuristic algorithm (FedAvgen), which relates each pretrained model with its weight space as metadata, to a phenotype and genotype, which characterizes the global averaging step in federated learning.

Abstract

To improve business efficiency and minimize costs, Artificial Intelligence (AI) practitioners have adopted a shift from formulating models from scratch towards sharing pretrained models. The pretrained models are then aggregated into a global model with higher generalization capabilities, which is afterwards distributed to the client devices. This approach is known as federated learning and inherently utilizes different techniques to select the candidate client models averaged to obtain the global model. This approach, in the case of communication systems, faces challenges arising from the existential diversity in device profiles. The multiplicity in profiles motivates our conceptual assessment of a metaheuristic algorithm (FedAvgen), which relates each pretrained model with its weight space as metadata, to a phenotype and genotype, respectively. This parent-child genetic evolution characterizes the global averaging step in federated learning. We then compare the results of our approach to two widely adopted baseline federated learning algorithms like Federated Averaging (FedAvg) and Federated Stochastic Gradient Descent (FedSGD).

FedAvgen: Metadata for Model Aggregation In Communication Systems

TL;DR

The multiplicity in profiles motivates the conceptual assessment of a metaheuristic algorithm (FedAvgen), which relates each pretrained model with its weight space as metadata, to a phenotype and genotype, which characterizes the global averaging step in federated learning.

Abstract

To improve business efficiency and minimize costs, Artificial Intelligence (AI) practitioners have adopted a shift from formulating models from scratch towards sharing pretrained models. The pretrained models are then aggregated into a global model with higher generalization capabilities, which is afterwards distributed to the client devices. This approach is known as federated learning and inherently utilizes different techniques to select the candidate client models averaged to obtain the global model. This approach, in the case of communication systems, faces challenges arising from the existential diversity in device profiles. The multiplicity in profiles motivates our conceptual assessment of a metaheuristic algorithm (FedAvgen), which relates each pretrained model with its weight space as metadata, to a phenotype and genotype, respectively. This parent-child genetic evolution characterizes the global averaging step in federated learning. We then compare the results of our approach to two widely adopted baseline federated learning algorithms like Federated Averaging (FedAvg) and Federated Stochastic Gradient Descent (FedSGD).
Paper Structure (10 sections, 10 equations, 3 figures, 2 tables)

This paper contains 10 sections, 10 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Federated Learning
  3. Goal
  4. Related Work
  5. Algorithm Design
  6. Genetic Algorithms
  7. Pre-trained Model Metadata
  8. Composing the Fitness Function
  9. Results and Evaluation
  10. Discussion and Conclusion

Figures (3)

  • Figure 1: The generalization capabilities of the FedAvgen algorithm against FedSGD with a lower final measure in the loss value.
  • Figure 2: In comparison to FedAvg, the FedAvgen algorithm gives better prediction capabilities.
  • Figure 3: The $FedAvgen$ here is less computationally intensive in comparison to $FedAvg$ but requires more processor cycles than the $FedSGD$ algorithm.