Determining the significance and relative importance of parameters of a simulated quenching algorithm using statistical tools

TL;DR

The ANOVA (ANalysis Of the VAriance) method is used to carry out an exhaustive analysis of a simulated annealing based method and the different parameters it requires and the adequacy of parameter values available in the bibliography is verified using parametric hypothesis tests.

Abstract

When search methods are being designed it is very important to know which parameters have the greatest influence on the behaviour and performance of the algorithm. To this end, algorithm parameters are commonly calibrated by means of either theoretic analysis or intensive experimentation. When undertaking a detailed statistical analysis of the influence of each parameter, the designer should pay attention mostly to the parameters that are statistically significant. In this paper the ANOVA (ANalysis Of the VAriance) method is used to carry out an exhaustive analysis of a simulated annealing based method and the different parameters it requires. Following this idea, the significance and relative importance of the parameters regarding the obtained results, as well as suitable values for each of these, were obtained using ANOVA and post-hoc Tukey HSD test, on four well known function optimization problems and the likelihood function that is used to estimate the parameters involved in the lognormal diffusion process. Through this statistical study we have verified the adequacy of parameter values available in the bibliography using parametric hypothesis tests.

Figures (10)

• Figure 1: TukeyHDS test results for Griewangk function. As can be seen, no significant differences between M and E schemes can be observed (the vertical line intersects the confidence segment corresponding to E-M). Likewise, the lowest value of IT is the most accurate, having found significant differences when compared to the others.
• Figure 2: As in the previous problem, TukeyHDS test for the Rastrigin function shows no significant differences between M and E schemes. Differences between the lowest value and the highest values of IT have been found. In the case of NC and NI differences appear when comparing extreme values, but not between close values (the vertical line crosses the confidence segments corresponding to comparisons between close values).
• Figure 3: TukeyHDS test results for Ackley function show clear differences between the E and M scheme regarding C; as in previous problem, in the case of NC and NI differences appear when comparing extreme values.
• Figure 4: TukeyHDS test results for Rana function. As in previous functions, no significant differences between M and E schemes can be observed (although clear differences can be found regarding C). Paying attention to NC, only small values present significant differences regarding the others (the vertical line does not intersect the confidence segments corresponding to these comparisons).
• Figure 5: Boxplot of the set of solutions grouping using each parameter for the Griewangk function. The distribution in the case of NC, NI and PS is similar, while there is a clear difference with CS and IT (the distribution is more asymmetric and focuses on those values whose average results are better).