Table of Contents
Fetching ...

A Prescription of Methodological Guidelines for Comparing Bio-inspired Optimization Algorithms

Antonio LaTorre, Daniel Molina, Eneko Osaba, Javier Del Ser, Francisco Herrera

Abstract

Bio-inspired optimization (including Evolutionary Computation and Swarm Intelligence) is a growing research topic with many competitive bio-inspired algorithms being proposed every year. In such an active area, preparing a successful proposal of a new bio-inspired algorithm is not an easy task. Given the maturity of this research field, proposing a new optimization technique with innovative elements is no longer enough. Apart from the novelty, results reported by the authors should be proven to achieve a significant advance over previous outcomes from the state of the art. Unfortunately, not all new proposals deal with this requirement properly. Some of them fail to select appropriate benchmarks or reference algorithms to compare with. In other cases, the validation process carried out is not defined in a principled way (or is even not done at all). Consequently, the significance of the results presented in such studies cannot be guaranteed. In this work we review several recommendations in the literature and propose methodological guidelines to prepare a successful proposal, taking all these issues into account. We expect these guidelines to be useful not only for authors, but also for reviewers and editors along their assessment of new contributions to the field.

A Prescription of Methodological Guidelines for Comparing Bio-inspired Optimization Algorithms

Abstract

Bio-inspired optimization (including Evolutionary Computation and Swarm Intelligence) is a growing research topic with many competitive bio-inspired algorithms being proposed every year. In such an active area, preparing a successful proposal of a new bio-inspired algorithm is not an easy task. Given the maturity of this research field, proposing a new optimization technique with innovative elements is no longer enough. Apart from the novelty, results reported by the authors should be proven to achieve a significant advance over previous outcomes from the state of the art. Unfortunately, not all new proposals deal with this requirement properly. Some of them fail to select appropriate benchmarks or reference algorithms to compare with. In other cases, the validation process carried out is not defined in a principled way (or is even not done at all). Consequently, the significance of the results presented in such studies cannot be guaranteed. In this work we review several recommendations in the literature and propose methodological guidelines to prepare a successful proposal, taking all these issues into account. We expect these guidelines to be useful not only for authors, but also for reviewers and editors along their assessment of new contributions to the field.

Paper Structure

This paper contains 41 sections, 9 figures, 19 tables.

Figures (9)

  • Figure 1: Different visualizations for the comparison of the performance of several algorithms on the CEC'2013 LSGO benchmark: (a) Average ranking of algorithms on different types of functions; (b) Fraction of functions for which a specific algorithm obtained the best results; (c) Fraction of multimodal functions for which a specific algorithm obtained the best results; (d) Fraction of shifted functions for which a specific algorithm obtained the best results.
  • Figure 2: Convergence curves for several functions of the CEC'2017 benchmark and dimension 10: (a) function 12; (b) function 28.
  • Figure 3: Box-plots for the CEC'2017 benchmark and dimension 30: (a) function 18; (b) function 26.
  • Figure 4: Critical distance plots for the CEC'2017 benchmark
  • Figure 5: Bayesian plots for the CEC'2017 benchmark and dimension 10.
  • ...and 4 more figures