Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Transfer for Evolutionary Multitasking

Sheng-Hao Wu, Yuxiao Huang, Xingyu Wu, Liang Feng, Zhi-Hui Zhan, Kay Chen Tan

TL;DR

A novel learning-to-transfer (L2T) framework to automatically discover efficient KT policies for the MTOPs at hand is proposed, which shows a marked improvement in the adaptability and performance of implicit EMT when solving a wide spectrum of unseen MTOPs.

Abstract

Evolutionary multitasking (EMT) is an emerging approach for solving multitask optimization problems (MTOPs) and has garnered considerable research interest. The implicit EMT is a significant research branch that utilizes evolution operators to enable knowledge transfer (KT) between tasks. However, current approaches in implicit EMT face challenges in adaptability, due to the use of a limited number of evolution operators and insufficient utilization of evolutionary states for performing KT. This results in suboptimal exploitation of implicit KT's potential to tackle a variety of MTOPs. To overcome these limitations, we propose a novel Learning to Transfer (L2T) framework to automatically discover efficient KT policies for the MTOPs at hand. Our framework conceptualizes the KT process as a learning agent's sequence of strategic decisions within the EMT process. We propose an action formulation for deciding when and how to transfer, a state representation with informative features of evolution states, a reward formulation concerning convergence and transfer efficiency gain, and the environment for the agent to interact with MTOPs. We employ an actor-critic network structure for the agent and learn it via proximal policy optimization. This learned agent can be integrated with various evolutionary algorithms, enhancing their ability to address a range of new MTOPs. Comprehensive empirical studies on both synthetic and real-world MTOPs, encompassing diverse inter-task relationships, function classes, and task distributions are conducted to validate the proposed L2T framework. The results show a marked improvement in the adaptability and performance of implicit EMT when solving a wide spectrum of unseen MTOPs.

Learning to Transfer for Evolutionary Multitasking

TL;DR

A novel learning-to-transfer (L2T) framework to automatically discover efficient KT policies for the MTOPs at hand is proposed, which shows a marked improvement in the adaptability and performance of implicit EMT when solving a wide spectrum of unseen MTOPs.

Abstract

Evolutionary multitasking (EMT) is an emerging approach for solving multitask optimization problems (MTOPs) and has garnered considerable research interest. The implicit EMT is a significant research branch that utilizes evolution operators to enable knowledge transfer (KT) between tasks. However, current approaches in implicit EMT face challenges in adaptability, due to the use of a limited number of evolution operators and insufficient utilization of evolutionary states for performing KT. This results in suboptimal exploitation of implicit KT's potential to tackle a variety of MTOPs. To overcome these limitations, we propose a novel Learning to Transfer (L2T) framework to automatically discover efficient KT policies for the MTOPs at hand. Our framework conceptualizes the KT process as a learning agent's sequence of strategic decisions within the EMT process. We propose an action formulation for deciding when and how to transfer, a state representation with informative features of evolution states, a reward formulation concerning convergence and transfer efficiency gain, and the environment for the agent to interact with MTOPs. We employ an actor-critic network structure for the agent and learn it via proximal policy optimization. This learned agent can be integrated with various evolutionary algorithms, enhancing their ability to address a range of new MTOPs. Comprehensive empirical studies on both synthetic and real-world MTOPs, encompassing diverse inter-task relationships, function classes, and task distributions are conducted to validate the proposed L2T framework. The results show a marked improvement in the adaptability and performance of implicit EMT when solving a wide spectrum of unseen MTOPs.
Paper Structure (31 sections, 21 equations, 5 figures, 3 tables, 2 algorithms)

This paper contains 31 sections, 21 equations, 5 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Evolutionary Multitasking
  4. Learning to Optimize
  5. Proposed Method
  6. L2T Workflow
  7. L2T Component Design
  8. MDP Formulation of KT Process
  9. Action Formulation
  10. State Representation
  11. Reward Design
  12. Agent and Environment
  13. Learning and Utilizing the Learnable Agent
  14. Learning Stage
  15. Utilization Stage
  16. ...and 16 more sections

Figures (5)

  • Figure 1: Diagram of the proposed L2T framework.
  • Figure 2: The first two dimensions of different global optimum distributions for defining tasks, which varies from the range of optimal solutions (a)-(e) and the number of distributed clusters (f)-(j).
  • Figure 3: Positive transfer rates of implicit EMT algorithms on problem sets with varied task optimum range (a) and number of clusters (b).
  • Figure 4: The training performance of agents by fine-tuning (L2T-FT) and retraining from scratch (L2T-w/o-FT) on new problem sets with significantly different task distributions.
  • Figure 5: The actions by the learned agent when solving two MTOP instances, MTOP15 (a)-(e) and MTOP23 (e)-(j) in the BBOB1 problem set.

Theorems & Definitions (1)

  • Definition 1