Monte Carlo Tree Search based Space Transfer for Black-box Optimization

TL;DR

This work tackles expensive black-box optimization by transferring and adaptively shaping the search space. It introduces MCTS-transfer, which builds a space-partitioning tree from source-task data in a pre-learning phase and then uses Monte Carlo Tree Search to navigate and refine a promising subspace during target-task optimization, weighting source tasks by similarity and updating the tree as new target data arrive. The approach yields warm-start subspaces, multiple promising regions, and automatic adaptation to source-target similarity, with demonstrated gains on BBOB benchmarks, Design-Bench problems, and HPOB, while maintaining modest runtime overhead. Overall, MCTS-transfer offers a general, flexible framework for transfer in BO that can be combined with GP-based surrogates and extended to more sophisticated acquisition strategies.

Abstract

Bayesian optimization (BO) is a popular method for computationally expensive black-box optimization. However, traditional BO methods need to solve new problems from scratch, leading to slow convergence. Recent studies try to extend BO to a transfer learning setup to speed up the optimization, where search space transfer is one of the most promising approaches and has shown impressive performance on many tasks. However, existing search space transfer methods either lack an adaptive mechanism or are not flexible enough, making it difficult to efficiently identify promising search space during the optimization process. In this paper, we propose a search space transfer learning method based on Monte Carlo tree search (MCTS), called MCTS-transfer, to iteratively divide, select, and optimize in a learned subspace. MCTS-transfer can not only provide a well-performing search space for warm-start but also adaptively identify and leverage the information of similar source tasks to reconstruct the search space during the optimization process. Experiments on synthetic functions, real-world problems, Design-Bench and hyper-parameter optimization show that MCTS-transfer can demonstrate superior performance compared to other search space transfer methods under different settings. Our code is available at \url{https://github.com/lamda-bbo/mcts-transfer}.

Figures (24)

• Figure 1: The workflow of MCTS-transfer. In pre-learning stage, MCTS-transfer builds the tree by clustering and classifying the samples apart recursively, until all nodes are not splitable. In optimization stage, the initial search space is based on the pre-learned tree. We will trace child node with greater UCB from ROOT and find the target leaf node to do optimization.
• Figure 2: Best value
• Figure 3: Weight change curves
• Figure 5: BBOB
• Figure 6: Real-world problems