Improving LLM-based Global Optimization with Search Space Partitioning

TL;DR

This work tackles the challenge of optimizing expensive blackbox functions with minimal prior knowledge by integrating large language models into a partitioned, hierarchical search framework. HOLLM adaptively partitions the input space using a KD-tree, assigns a bandit-inspired score to each region that fuses exploitation, geometry, and uncertainty, and uses LLMs to generate localized candidate points within selected regions. The approach combines nonparametric bandit ideas with LLM-based sampling to overcome high-dimensional and multimodal landscapes, achieving performance on par with or better than leading Bayesian optimization and trust-region methods across synthetic tasks, hyperparameter tuning, and neural architecture search. The results indicate HOLLM’s potential to enhance LLM-driven optimization in practice, while the authors acknowledge limitations such as the lack of formal regret guarantees, dependency on LLM quality, and the computational costs of inference. This method offers a scalable path toward more reliable and efficient LLM-guided optimization in complex scientific and engineering domains.

Abstract

Large Language Models (LLMs) have recently emerged as effective surrogate models and candidate generators within global optimization frameworks for expensive blackbox functions. Despite promising results, LLM-based methods often struggle in high-dimensional search spaces or when lacking domain-specific priors, leading to sparse or uninformative suggestions. To overcome these limitations, we propose HOLLM, a novel global optimization algorithm that enhances LLM-driven sampling by partitioning the search space into promising subregions. Each subregion acts as a ``meta-arm'' selected via a bandit-inspired scoring mechanism that effectively balances exploration and exploitation. Within each selected subregion, an LLM then proposes high-quality candidate points, without any explicit domain knowledge. Empirical evaluation on standard optimization benchmarks shows that HOLLM consistently matches or surpasses leading Bayesian optimization and trust-region methods, while substantially outperforming global LLM-based sampling strategies.

Figures (10)

• Figure 1: (a) 80 samples in $[0,1]^2$: Gemini-1.5 simulating uniform sampling (green), and with region-wise partitioning (red) using the prompt in Listing \ref{['lst:prompt-uniform']}. (b) Gemini-1.5 prompted (see Listing \ref{['lst:prompt-quadratic']}) to generate 80 samples around the 2 minima (red crosses) globally (triangles) and withing the two bounding boxes (circles). (c) Hausdorff distance $d_H(\mathcal{P},[0,1]^8)$ for uniform vs. LLM-simulated sampling in the 8-D hypercube.
• Figure 2: Overview of the HOLLM algorithm: starting from initial data $\mathcal{D}$, it iteratively performs Partition, Score, Select, Sample (via LLM), and Evaluate steps to balance exploration and exploitation. For the partitioning here, we utilized a KD-Tree where each axis is split based on the mean values. Each rectangle represents a partition defined by the tree leaves. The red stars represent the new sampled points from the LLM.
• Figure 3: Best function value across 100 iterations on the synthetic problems. HOLLM outperforms or matches the performance of baselines, especially on higher dimensional problems (e.g., Ackley).
• Figure 4: Illustrative example of HOLLM optimizing a 1D multimodal problem. The rectangles represent the space partitions (top figure) and are highlighted in orange whenever they are selected based on their respective probabilities (bottom figure). We used a batch size of 3. All new points (red stars) are LLM suggestions. Notice how partitions become fine-grained around the global maximum.
• Figure 5: Hyperparameter optimization on 4 datasets from the FCNet search space. All baselines from Synetune are evaluated asynchronously using 4 workers.