Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MeGA: Merging Multiple Independently Trained Neural Networks Based on Genetic Algorithm

Daniel Yun

TL;DR

MeGA addresses the problem of merging multiple independently trained CNNs with identical architectures by optimizing weight fusion with a genetic algorithm. It defines a GA framework with initialization, tournament selection, crossover, and mutation, using validation accuracy as the fitness to guide evolution, and supports hierarchical multi-model merging. Experiments on CIFAR-10 show MeGA consistently improves final test accuracy over individual models and simple weight averaging across ResNet, Xception, and DenseNet families, with hierarchical merging of eight models further demonstrating scalability. The approach enables efficient, distributed-friendly integration of pre-trained networks, providing a scalable tool for enhancing neural network performance without additional training or architectural changes.

Abstract

In this paper, we introduce a novel method for merging the weights of multiple pre-trained neural networks using a genetic algorithm called MeGA. Traditional techniques, such as weight averaging and ensemble methods, often fail to fully harness the capabilities of pre-trained networks. Our approach leverages a genetic algorithm with tournament selection, crossover, and mutation to optimize weight combinations, creating a more effective fusion. This technique allows the merged model to inherit advantageous features from both parent models, resulting in enhanced accuracy and robustness. Through experiments on the CIFAR-10 dataset, we demonstrate that our genetic algorithm-based weight merging method improves test accuracy compared to individual models and conventional methods. This approach provides a scalable solution for integrating multiple pre-trained networks across various deep learning applications. Github is available at: https://github.com/YUNBLAK/MeGA-Merging-Multiple-Independently-Trained-Neural-Networks-Based-on-Genetic-Algorithm

MeGA: Merging Multiple Independently Trained Neural Networks Based on Genetic Algorithm

TL;DR

MeGA addresses the problem of merging multiple independently trained CNNs with identical architectures by optimizing weight fusion with a genetic algorithm. It defines a GA framework with initialization, tournament selection, crossover, and mutation, using validation accuracy as the fitness to guide evolution, and supports hierarchical multi-model merging. Experiments on CIFAR-10 show MeGA consistently improves final test accuracy over individual models and simple weight averaging across ResNet, Xception, and DenseNet families, with hierarchical merging of eight models further demonstrating scalability. The approach enables efficient, distributed-friendly integration of pre-trained networks, providing a scalable tool for enhancing neural network performance without additional training or architectural changes.

Abstract

In this paper, we introduce a novel method for merging the weights of multiple pre-trained neural networks using a genetic algorithm called MeGA. Traditional techniques, such as weight averaging and ensemble methods, often fail to fully harness the capabilities of pre-trained networks. Our approach leverages a genetic algorithm with tournament selection, crossover, and mutation to optimize weight combinations, creating a more effective fusion. This technique allows the merged model to inherit advantageous features from both parent models, resulting in enhanced accuracy and robustness. Through experiments on the CIFAR-10 dataset, we demonstrate that our genetic algorithm-based weight merging method improves test accuracy compared to individual models and conventional methods. This approach provides a scalable solution for integrating multiple pre-trained networks across various deep learning applications. Github is available at: https://github.com/YUNBLAK/MeGA-Merging-Multiple-Independently-Trained-Neural-Networks-Based-on-Genetic-Algorithm
Paper Structure (17 sections, 7 equations, 3 figures, 2 tables, 1 algorithm)

This paper contains 17 sections, 7 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Genetic Algorithm Framework
  5. Preliminaries.
  6. Algorithm Execution.
  7. Initialization
  8. Fitness Evaluation
  9. Selection
  10. Crossover
  11. Mutation
  12. Experimental Results
  13. Experimental Settings
  14. CIFAR-10 Results
  15. Extended Experiments for Multi-Models
  16. ...and 2 more sections

Figures (3)

  • Figure 1: MeGA: The process of merging two neural networks using a genetic algorithm.
  • Figure 2: MeGA Fitness Progression (Test Accuracy) for ResNet-56 and DenseNet 121 models over 20 generations.
  • Figure 3: Hierarchical merging process of eight models into a single final model.