COBRA++: Enhanced COBRA Optimizer with Augmented Surrogate Pool and Reinforced Surrogate Selection

TL;DR

Addressing the challenge of expensive constrained black-box optimization, the paper introduces COBRA++, which extends COBRA with an augmented pool of $RBF$ surrogates and a reinforcement learning-based online surrogate selection policy. The RL agent operates on an $MDP$ that selects among 11 diverse $RBF$ kernels to guide the lower-level COBRA optimizer, while retraining all surrogates after each true evaluation to maintain up-to-date predictions. Trained across a distribution of problems, the policy aims to maximize cumulative optimization performance, with a Deep Q-Network architecture featuring separate feature extractors for surrogate- and global-state information. Empirical results on the COCO bbob-constrained benchmark show substantial improvements over vanilla COBRA and related variants in both 10D and 40D settings, supported by ablation studies that validate each component and by analyses of phase-specific surrogate usage, indicating robust generalization and potential for future multi-objective extensions.

Abstract

The optimization problems in realistic world present significant challenges onto optimization algorithms, such as the expensive evaluation issue and complex constraint conditions. COBRA optimizer (including its up-to-date variants) is a representative and effective tool for addressing such optimization problems, which introduces 1) RBF surrogate to reduce online evaluation and 2) bi-stage optimization process to alternate search for feasible solution and optimal solution. Though promising, its design space, i.e., surrogate model pool and selection standard, is still manually decided by human expert, resulting in labor-intensive fine-tuning for novel tasks. In this paper, we propose a learning-based adaptive strategy (COBRA++) that enhances COBRA in two aspects: 1) An augmented surrogate pool to break the tie with RBF-like surrogate and hence enhances model diversity and approximation capability; 2) A reinforcement learning-based online model selection policy that empowers efficient and accurate optimization process. The model selection policy is trained to maximize overall performance of COBRA++ across a distribution of constrained optimization problems with diverse properties. We have conducted multi-dimensional validation experiments and demonstrate that COBRA++ achieves substantial performance improvement against vanilla COBRA and its adaptive variant. Ablation studies are provided to support correctness of each design component in COBRA++.

Figures (5)

• Figure 1: Overview of COBRA++.
• Figure 2: Network Architecture of COBRA++.
• Figure 3: Performance Comparison on 40D Problems
• Figure 4: Return curve of COBRA++.
• Figure 5: Model selection frequency across three optimization phases