A Theoretical Analysis of Analogy-Based Evolutionary Transfer Optimization

TL;DR

This work addresses the theoretical gap in analogy-based evolutionary transfer optimization (ETO) by formalizing analogical reasoning and its three subprocesses—retrieval, mapping, and evaluation—and linking them to the core ETO questions. It introduces the theoretical foundations for analogy-based knowledge transfer and proves two theorems: unconditionally nonnegative performance gain and conditionally positive performance gain, establishing conditions under which transfer is safe or beneficial. A novel NFL-based perspective shows that the conditional superiority of ETO arises when analogy-based biases align more closely with the problem class than general EO biases. These results provide a principled basis for designing interpretable, generalizable analogy-based ETO algorithms and motivate empirical work to automatically discover and align analogy-based biases with specific problems.

Abstract

Evolutionary transfer optimization (ETO) has been gaining popularity in research over the years due to its outstanding knowledge transfer ability to address various challenges in optimization. However, a pressing issue in this field is that the invention of new ETO algorithms has far outpaced the development of fundamental theories needed to clearly understand the key factors contributing to the success of these algorithms for effective generalization. In response to this challenge, this study aims to establish theoretical foundations for analogy-based ETO, specifically to support various algorithms that frequently reference a key concept known as similarity. First, we introduce analogical reasoning and link its subprocesses to three key issues in ETO. Then, we develop theories for analogy-based knowledge transfer, rooted in the principles that underlie the subprocesses. Afterwards, we present two theorems related to the performance gain of analogy-based knowledge transfer, namely unconditionally nonnegative performance gain and conditionally positive performance gain, to theoretically demonstrate the effectiveness of various analogy-based ETO methods. Last but not least, we offer a novel insight into analogy-based ETO that interprets its conditional superiority over traditional evolutionary optimization through the lens of the no free lunch theorem for optimization.

Figures (15)

• Figure 1: Analogy-based rationale for knowledge transfer.
• Figure 2: Illustrations of some monotonic functions $f:S\to U$ and the retrieval process in Eq. \ref{['eq:retrieval']}: (a) the monotonic functions; (b) the retrieval process.
• Figure 3: An illustrative comparison between explicit mapping learning and mixture-based implicit mapping: (a) the explicit mapping; (b) the mixture-based mapping.
• Figure 4: An illustration of the mapping process in Eq. \ref{['eq:mapping']}.
• Figure 5: An illustrative comparison between explicit similarity assessment and mixture-based implicit adaptive transfer: (a) the explicit similarity assessment; (b) the mixture-based adaptive transfer.