SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality

TL;DR

Bayesian optimization suffers from a curse of dimensionality due to GP-based surrogates with $O(N^3)$ scaling as data and dimensionality grow. The authors introduce SCORE, a 1D reparameterization that projects a $D$-dimensional space onto per-parameter axes and runs GP regression in 1D, aiming to preserve linear-time behavior and enable faster convergence. They demonstrate SCORE on high-dimensional Ackley functions (up to $200$D) and on fitting a solar single-diode model to IV data, achieving global minima or high-quality fits with far fewer evaluations on a standard laptop. A Python implementation is provided, suggesting SCORE can be integrated with existing BO frameworks to accelerate high-dimensional, expensive optimization tasks.

Abstract

Bayesian optimization (BO) has emerged as a powerful tool for navigating complex search spaces, showcasing practical applications in the fields of science and engineering.However, since it typically relies on a surrogate model to approximate the objective function, BO grapples with heightened computational costs that tend to escalate as the number of parameters and experiments grows. Several methods such as parallelization, surrogate model approximations, and memory pruning have been proposed to cut down computing time, but they all fall short of resolving the core issue behind BO's curse of dimensionality. In this paper, a 1D reparametrization trick is proposed to break this curse and sustain linear time complexity for BO in high-dimensional landscapes. This fast and scalable approach named SCORE can successfully find the global minimum of needle-in-a-haystack optimization functions and fit real-world data without the high-performance computing resources typically required by state-of-the-art techniques.

Figures (5)

• Figure 1: Convergence plots of Bayesian Optimization and two $SCORE$ configurations on the 10D Ackley function.
• Figure 2: Average computing times of Bayesian Optimization and two $SCORE$ configurations on the 10D Ackley function.
• Figure 3: Computing times of Bayesian Optimization and $SCORE$ as a function of the number of iterations on the 5D Ackley function.
• Figure 4: Convergence plots of two $SCORE$ configurations on the 200D Ackley function.
• Figure 5: Measured data vs. fits obtained by Bayesian Optimization and two $SCORE$ configurations.