Black-Box Approximation and Optimization with Hierarchical Tucker Decomposition

TL;DR

The paper addresses the challenge of surrogate modeling and gradient-free optimization for high-dimensional black-box functions by leveraging the hierarchical Tucker (HT) tensor decomposition. It introduces HTBB, a unified framework comprising HT-cross for approximation and HTOpt for optimization, both powered by the rectangular MaxVol index selection and a sequential core-traversal scheme. The authors demonstrate substantial accuracy and robustness advantages over TT-based surrogates (e.g., TT-cross) and classical optimizers (e.g., SPSA, PSO) on 14 complex benchmarks up to $d=1000$, with a fixed query budget. The work advances scalable high-dimensional BB handling and provides a public Python package, highlighting HT’s expressive power and practical impact in engineering and ML settings where gradients are unavailable.

Abstract

We develop a new method HTBB for the multidimensional black-box approximation and gradient-free optimization, which is based on the low-rank hierarchical Tucker decomposition with the use of the MaxVol indices selection procedure. Numerical experiments for 14 complex model problems demonstrate the robustness of the proposed method for dimensions up to 1000, while it shows significantly more accurate results than classical gradient-free optimization methods, as well as approximation and optimization methods based on the popular tensor train decomposition, which represents a simpler case of a tensor network.

Figures (5)

• Figure 1: Examples of upper and down indices and their values for $\mathcal{Y}\in\mathbb R^{ N_1\times N_2\times N_3\times N_4\times N_5\times N_6\times N_7\times N_8 }$ with $N_1=N_2=N_5=N_6=N_7=2$, $N_3=N_4=3$, and $N_8=10$.
• Figure 2: Algorithm \ref{['alg:update idx']} inputs for the cases of upper and down indices values update. On the left: when updating upwards, the indices forming the rows of $A$ are calculated based on the upper indices on the links below ($i_1$ and $i_2$) and their values, and the indices $i$ (and their values $v$) forming the row of the matrix $A$ consists of the down indices of the link above and their values. The values of the upper indices associated with the link above are updated. On the right: similar updating but with slight changes occurs when moving downwards.
• Figure 3: Examples of a path for the traversal procedure. The task is 5-dimensional, so indices 5, 6, and 7 (green boxes) as well as their parents (purple circles) are never visited. Ranks of all links, except for those leading to inactive indices, are equal to 3.
• Figure 4: Approximation results for Alpine and Dixon functions for cases of dimensions $5$, $10$, $50$, $100$, and $200$. For both methods, we plot the relative error of the solution averaged over $10$ runs with a solid line and fill in the area between the worst and best result with the same color.
• Figure 5: Minimization results for Alpine and Dixon functions. For each of the optimizers, we plot the value of the solution averaged over $10$ runs with a solid line and fill in the area between the worst and best result with the same color.