Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A partitioned optimization framework for structure-aware optimization

Charles Audet, Pierre-Yves Bouchet, Loïc Bourdin

TL;DR

This work introduces the partitioned optimization framework (POf), which partitions the decision space into subsets that render restricted subproblems tractable, and reformulates the original problem as a low-dimensional optimization over partition indices via the oracle $\\gamma$ and the objective $\\Φ(x)=\\varphi(\\gamma(x))$. It then proposes the derivative-free partitioned optimization method (DFPOm), which uses a covering-step derivative-free algorithm to solve the reformulated problem and recovers a solution to the original problem from the optimal partition index. The theoretical contributions guarantee that solutions to the reformulated problem map to (generalized) local solutions of the original problem under mild assumptions, and the practical framework is demonstrated on composite greybox problems with large-scale instances where it outperforms standard DFO solvers like NOMAD and PRIMA. The results show substantial numerical gains by exploiting structure through partitioning, with broad implications for structure-aware optimization in high-dimensional settings and potential extensions to adaptive partitioning and relaxed oracle guarantees. Overall, the paper provides a principled, theory-backed pathway to leverage problem structure for efficient, scalable derivative-free optimization.

Abstract

This work tackles a class of optimization problems where fixing some well-chosen combinations of the variables makes the problem substantially easier to solve. We consider that the variables space may be partitioned into subsets that fix these combinations to given values, so the restriction of the problem to any of the partition sets admits a tractable solution. Then, we exhibit a reformulation of the original problem that consists in searching for the partition set index that minimizes the objective value of the solution to the restricted problem. We name partitioned optimization framework (POf) the formalization of this class of problems and this reformulated problem. As we prove in this work, the POf allows solving the original problem by focusing on the reformulated problem: all solutions to the reformulated problem are partition indices for which the solution to the associated restricted problem is also a solution to the original problem. Second, we introduce a derivative-free partitioned optimization method (DFPOm) to efficiently solve problems that fit in the POf. We prove that the reformulated problem is nicely handled by a class of derivative-free optimization (DFO) algorithms named algorithms with a covering step. Then the DFPOm consists in solving the reformulated problem using such DFO algorithm with a covering step to obtain an optimal partition index, and to return the solution to the associated restricted problem as a solution to the initial problem. Finally, we illustrate the DFPOm on a class of problems called composite greybox problems, and we highlight the gain in numerical performance provided by the DFPOm on large-dimensional instances by comparing it to two popular DFO solvers.

A partitioned optimization framework for structure-aware optimization

TL;DR

This work introduces the partitioned optimization framework (POf), which partitions the decision space into subsets that render restricted subproblems tractable, and reformulates the original problem as a low-dimensional optimization over partition indices via the oracle and the objective . It then proposes the derivative-free partitioned optimization method (DFPOm), which uses a covering-step derivative-free algorithm to solve the reformulated problem and recovers a solution to the original problem from the optimal partition index. The theoretical contributions guarantee that solutions to the reformulated problem map to (generalized) local solutions of the original problem under mild assumptions, and the practical framework is demonstrated on composite greybox problems with large-scale instances where it outperforms standard DFO solvers like NOMAD and PRIMA. The results show substantial numerical gains by exploiting structure through partitioning, with broad implications for structure-aware optimization in high-dimensional settings and potential extensions to adaptive partitioning and relaxed oracle guarantees. Overall, the paper provides a principled, theory-backed pathway to leverage problem structure for efficient, scalable derivative-free optimization.

Abstract

This work tackles a class of optimization problems where fixing some well-chosen combinations of the variables makes the problem substantially easier to solve. We consider that the variables space may be partitioned into subsets that fix these combinations to given values, so the restriction of the problem to any of the partition sets admits a tractable solution. Then, we exhibit a reformulation of the original problem that consists in searching for the partition set index that minimizes the objective value of the solution to the restricted problem. We name partitioned optimization framework (POf) the formalization of this class of problems and this reformulated problem. As we prove in this work, the POf allows solving the original problem by focusing on the reformulated problem: all solutions to the reformulated problem are partition indices for which the solution to the associated restricted problem is also a solution to the original problem. Second, we introduce a derivative-free partitioned optimization method (DFPOm) to efficiently solve problems that fit in the POf. We prove that the reformulated problem is nicely handled by a class of derivative-free optimization (DFO) algorithms named algorithms with a covering step. Then the DFPOm consists in solving the reformulated problem using such DFO algorithm with a covering step to obtain an optimal partition index, and to return the solution to the associated restricted problem as a solution to the initial problem. Finally, we illustrate the DFPOm on a class of problems called composite greybox problems, and we highlight the gain in numerical performance provided by the DFPOm on large-dimensional instances by comparing it to two popular DFO solvers.
Paper Structure (28 sections, 11 theorems, 37 equations, 15 figures, 4 tables, 2 algorithms)

This paper contains 28 sections, 11 theorems, 37 equations, 15 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Abstract:
  2. Acknowledgements:
  3. Introduction
  4. Illustrative example with a composite greybox problem
  5. Motivation for this work
  6. Contribution and outline of this work
  7. Review of related literature
  8. Formal framework and related theoretical results
  9. Formal definition of the POf and theoretical properties
  10. The DFPOm and related convergence certifications
  11. Proofs of Theorems \ref{['theorem:connections_solutions']} and \ref{['theorem:solving_P']}
  12. Technical lemmas on the problem, the framework and its minimal assumption
  13. Proof of Theorem \ref{['theorem:connections_solutions']}
  14. Proof of validity of Proposition \ref{['proposition:cDSM_applied_to_Phi']} under Assumption \ref{['assumption:pof_additional']}
  15. Proof of Theorem \ref{['theorem:solving_P']}
  16. ...and 13 more sections

Key Result

Theorem 1

In Framework framework:PO under Assumption assumption:pof_minimal, the next three statements hold:

Figures (15)

  • Figure 1: The function $\varphi$ from Sections \ref{['section:intro/motivation']} and \ref{['section:applications/monovariable']}, minimized efficiently via the POf. For readability, at each $y_1 \in [-4,4]$ the two graphs actually plot only the values of the quadratic $y_2 \mapsto \varphi(y_1,y_2)$ for $y_2 \in [\sigma(y_1)-1.3,\ \sigma(y_1)+1.3]$.
  • Figure 2: Function $\varphi$ in Section \ref{['section:applications/monovariable']}, (left) 3D view, (right) 2D view, partition of $\mathbb{Y}$ and locus of $\gamma$. For each $y_1 \in \mathbb{R}$, we plot $\varphi_{|\{y_1\}\times\mathbb{R}}$ for the values $y_2 \in [\sigma(y_1)-1.3,\ \sigma(y_1)+1.3]$ only.
  • Figure 3: Functions $\varepsilon$ and $\Phi$ in Section \ref{['section:applications/monovariable']}, (left) function $\varepsilon$ and its components, (right) function $\Phi$. In this section, $\Phi = \varepsilon$.
  • Figure 4: Function $\varphi$ in Section \ref{['section:applications/radial']}, (left) 3D view, (right) 2D view, partition of $\mathbb{Y}$ and locus of $\gamma$.
  • Figure 5: Functions $\varepsilon$ and $\Phi$ in Section \ref{['section:applications/radial']}, (left) function $\varepsilon$ and its components, (right) function $\Phi$. In this section, $\Phi = \varepsilon$.
  • ...and 10 more figures

Theorems & Definitions (18)

  • Definition 1: Generalized local solutions
  • Theorem 1
  • Remark 1
  • Remark 2
  • Definition 2
  • Proposition 1: convergence analysis of Algorithm \ref{['algo:covering_framework']}
  • Theorem 2
  • Remark 3
  • Remark 4
  • Remark 5
  • ...and 8 more