Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning from Offline and Online Experiences: A Hybrid Adaptive Operator Selection Framework

Jiyuan Pei, Jialin Liu, Yi Mei

TL;DR

This work tackles the challenge of learning to adapt operator selection in meta-heuristics by explicitly hybridising offline experiences from past problems with online experiences gathered during solving a new problem. It introduces a dual AOS setup—a state-based module trained on historical data plus online updates, and a lightweight stateless module trained from online data—with a dynamic decision policy to balance their use. Across real-valued benchmarks and a challenging CVRPTW instance set, the proposed hybrid framework outperforms state-of-the-art state-based and stateless AOS methods, with ablations confirming the benefits of online updating, module cooperation, and adaptive weighting. The results suggest substantial practical impact for solving repeated optimisation tasks in uncertain, resource-constrained settings, and indicate paths for extending the approach to broader problem classes and more sophisticated state representations.

Abstract

In many practical applications, usually, similar optimisation problems or scenarios repeatedly appear. Learning from previous problem-solving experiences can help adjust algorithm components of meta-heuristics, e.g., adaptively selecting promising search operators, to achieve better optimisation performance. However, those experiences obtained from previously solved problems, namely offline experiences, may sometimes provide misleading perceptions when solving a new problem, if the characteristics of previous problems and the new one are relatively different. Learning from online experiences obtained during the ongoing problem-solving process is more instructive but highly restricted by limited computational resources. This paper focuses on the effective combination of offline and online experiences. A novel hybrid framework that learns to dynamically and adaptively select promising search operators is proposed. Two adaptive operator selection modules with complementary paradigms cooperate in the framework to learn from offline and online experiences and make decisions. An adaptive decision policy is maintained to balance the use of those two modules in an online manner. Extensive experiments on 170 widely studied real-value benchmark optimisation problems and a benchmark set with 34 instances for combinatorial optimisation show that the proposed hybrid framework outperforms the state-of-the-art methods. Ablation study verifies the effectiveness of each component of the framework.

Learning from Offline and Online Experiences: A Hybrid Adaptive Operator Selection Framework

TL;DR

This work tackles the challenge of learning to adapt operator selection in meta-heuristics by explicitly hybridising offline experiences from past problems with online experiences gathered during solving a new problem. It introduces a dual AOS setup—a state-based module trained on historical data plus online updates, and a lightweight stateless module trained from online data—with a dynamic decision policy to balance their use. Across real-valued benchmarks and a challenging CVRPTW instance set, the proposed hybrid framework outperforms state-of-the-art state-based and stateless AOS methods, with ablations confirming the benefits of online updating, module cooperation, and adaptive weighting. The results suggest substantial practical impact for solving repeated optimisation tasks in uncertain, resource-constrained settings, and indicate paths for extending the approach to broader problem classes and more sophisticated state representations.

Abstract

In many practical applications, usually, similar optimisation problems or scenarios repeatedly appear. Learning from previous problem-solving experiences can help adjust algorithm components of meta-heuristics, e.g., adaptively selecting promising search operators, to achieve better optimisation performance. However, those experiences obtained from previously solved problems, namely offline experiences, may sometimes provide misleading perceptions when solving a new problem, if the characteristics of previous problems and the new one are relatively different. Learning from online experiences obtained during the ongoing problem-solving process is more instructive but highly restricted by limited computational resources. This paper focuses on the effective combination of offline and online experiences. A novel hybrid framework that learns to dynamically and adaptively select promising search operators is proposed. Two adaptive operator selection modules with complementary paradigms cooperate in the framework to learn from offline and online experiences and make decisions. An adaptive decision policy is maintained to balance the use of those two modules in an online manner. Extensive experiments on 170 widely studied real-value benchmark optimisation problems and a benchmark set with 34 instances for combinatorial optimisation show that the proposed hybrid framework outperforms the state-of-the-art methods. Ablation study verifies the effectiveness of each component of the framework.
Paper Structure (19 sections, 1 equation, 2 figures, 10 tables, 1 algorithm)

This paper contains 19 sections, 1 equation, 2 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Adaptive Operator Selection
  4. Generalisation in Optimisation with Learning
  5. Hybrid Adaptive Operator Selection Framework
  6. Experimental study
  7. Experiment Setting
  8. Training and test sets
  9. Evaluation metric
  10. Compared methods
  11. Implementation of hybrid framework
  12. Effectiveness of the Hybrid AOS Framework
  13. Performance on test set
  14. Performance on training set
  15. Ablation Study
  16. ...and 4 more sections

Figures (2)

  • Figure 1: The proposed hybrid AOS framework.
  • Figure 2: Average objective value and standard deviation of solutions obtained on two real-value problems (top) and two Solomon CVRPTW instances (bottom).