Global Optimization via Optimal Decision Trees

TL;DR

This work addresses global optimization with arbitrary explicit and inexplicit constraints by learning MIO-compatible approximations using optimal decision trees with hyperplanes (OCT-Hs). The authors build a three-step pipeline: transform the problem to standard form, sample nonlinear constraints to train OCT-Hs, and solve the resulting disjunctive MI approximation, followed by a projection-based repair to ensure feasibility and local optimality. Key innovations include disciplined constraint sampling, a big-$M$-free disjunctive MILP representation, and a gradient-descent repair mechanism that leverages auto-differentiation for feasibility restoration. Empirically, OCT-HaGOn demonstrates competitive performance on MINLPLib benchmarks and realistic aerospace problems (e.g., Speed Reducer and Satellite OOS), highlighting its ability to handle black-box and data-driven constraints while offering interpretability and computational tractability for a broad class of global optimization problems.

Abstract

The global optimization literature places large emphasis on reducing intractable optimization problems into more tractable structured optimization forms. In order to achieve this goal, many existing methods are restricted to optimization over explicit constraints and objectives that use a subset of possible mathematical primitives. These are limiting in real-world contexts where more general explicit and black box constraints appear. Leveraging the dramatic speed improvements in mixed-integer optimization (MIO) and recent research in machine learning, we propose a new method to learn MIO-compatible approximations of global optimization problems using optimal decision trees with hyperplanes (OCT-Hs). This constraint learning approach only requires a bounded variable domain, and can address both explicit and inexplicit constraints. We solve the MIO approximation efficiently to find a near-optimal, near-feasible solution to the global optimization problem. We further improve the solution using a series of projected gradient descent iterations. We test the method on a number of numerical benchmarks from the literature as well as real-world design problems, demonstrating its promise in finding global optima efficiently.

Figures (9)

• Figure 1: The distribution of data for constraint $g_1({\bf x}) \geq 0$, generated by sampling procedures defined in Section \ref{['sec:sampling']}.
• Figure 2: The approximating OCT-H achieves a high degree of accuracy, capturing both the global and local behavior of the constraint $g_1({\bf x}) \geq 0$.
• Figure 3: $g_1({\bf x}) \geq 0$ is approximated via 6 continuous and 2 binary auxiliary variables, and 6 linear constraints.
• Figure 4: $g_2({\bf x}) \geq 0$ is approximated via 8 continuous and 2 binary auxiliary variables, and 7 linear constraints.
• Figure 5: The MIO solution to the demonstrative example is successfully repaired to be feasible and locally optimal by the PGD method.