GEGO: A Hybrid Golden Eagle and Genetic Optimization Algorithm for Efficient Hyperparameter Tuning in Resource-Constrained Environments
Amaras Nazarians, Sachin Kumar
TL;DR
GEGO tackles the high cost of hyperparameter tuning in neural networks by marrying Golden Eagle Optimization with Genetic Algorithms, embedding crossover and mutation directly into GEO's iterative search to sustain population diversity. The hybrid maintains GEO's directed exploration while injecting evolutionary diversity, and it is evaluated on CEC2017 benchmarks and neural network tuning on MNIST under limited computational budgets. GEGO consistently improves solution quality and convergence stability compared to its constituents (GEO and GA) and several baselines, achieving near-top performance on complex functions and superior MNIST test accuracy (up to 97.90%). The results indicate GEGO's effective balance of exploration and exploitation in mixed continuous–discrete search spaces, with practical implications for resource-constrained hyperparameter optimization in real-world ML pipelines, and they justify further exploration at larger scales and in more diverse tasks. The computational footprint remains dominated by fitness evaluations, with GEGO adding only constant-factor overhead to the underlying GEO dynamics, preserving scalability in budgets where training costs dwarf optimizer costs.
Abstract
Hyperparameter tuning is a critical yet computationally expensive step in training neural networks, particularly when the search space is high dimensional and nonconvex. Metaheuristic optimization algorithms are often used for this purpose due to their derivative free nature and robustness against local optima. In this work, we propose Golden Eagle Genetic Optimization (GEGO), a hybrid metaheuristic that integrates the population movement strategy of Golden Eagle Optimization with the genetic operators of selection, crossover, and mutation. The main novelty of GEGO lies in embedding genetic operators directly into the iterative search process of GEO, rather than applying them as a separate evolutionary stage. This design improves population diversity during search and reduces premature convergence while preserving the exploration behavior of GEO. GEGO is evaluated on standard unimodal, multimodal, and composite benchmark functions from the CEC2017 suite, where it consistently outperforms its constituent algorithms and several classical metaheuristics in terms of solution quality and robustness. The algorithm is further applied to hyperparameter tuning of artificial neural networks on the MNIST dataset, where GEGO achieves improved classification accuracy and more stable convergence compared to GEO and GA. These results indicate that GEGO provides a balanced exploration-exploitation tradeoff and is well suited for hyperparameter optimization under constrained computational settings.