Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On leveraging constrained smooth additive regression models for global optimization

Marina Cuesta, Claudia D'Ambrosio, María Durban, Vanesa Guerrero, Renan Spencer Trindade

TL;DR

The paper addresses the difficulty of solving complex MINLPs by constructing surrogate problems through constrained smoothing additive regression using $B$-splines, enabling a data-driven and expert-informed approximation of nonlinear terms. MiSSOC replaces complicating functions with additive spline surrogates and solves the surrogate MINLP—best when tackled with the $SC$-MINLP algorithm, followed by local NLP refinement—demonstrating strong performance on MINLPlib benchmarks and a Hydro Unit Commitment case study. The results show that MiSSOC yields high-quality solutions with significantly lower computational effort than solving the original problems directly, and can even surpass the best known feasible solutions in real-world instances. This work highlights the practical viability of combining data-driven surrogates with shape and bound constraints to tackle large, nonconvex MINLPs.

Abstract

Many real-world decision-making processes rely on solving mixed-integer nonlinear programming (MINLP) problems. However, finding high-quality solutions to MINLPs is often computationally demanding. This has motivated the development of specialized algorithms that take advantage of the structure of some MINLPs to improve their tractability. In this work, we propose the Mixed-integer Smoothing Surrogate Optimization with Constraints (MiSSOC) approach, a novel optimization algorithm that builds and solves surrogate problems for complex MINLPs which are more tractable in practice. MiSSOC integrates statistical modeling into mathematical optimization by approximating complex functions in an MINLP using smooth additive regression models with $B-$splines. Expert knowledge can be incorporated into the building phase of the approximating functions through shape constraints related to sign (or bounds), monotonicity and convexity over the observed domain. Thus, MiSSOC fills a gap in the literature by building surrogates that are both data-driven and knowledge-driven. The surrogate problem is formulated by replacing the original complex functions with their approximations. The proposed MiSSOC algorithm is evaluated through a set of experiments, including benchmark instances and a real-world case study. MiSSOC is tested with different state-of-the-art solvers and with the tailored Sequential Convex MINLP (SC-MINLP) algorithm. The latter exploits the separable structure of the surrogate functions, which results from the approximating functions being sums of piecewise univariate polynomials. MiSSOC is benchmarked against solving the original MINLPs directly. The results show that MiSSOC is an effective approach for addressing complex MINLPs through surrogate modeling, particularly when used in combination with the SC-MINLP algorithm.

On leveraging constrained smooth additive regression models for global optimization

TL;DR

The paper addresses the difficulty of solving complex MINLPs by constructing surrogate problems through constrained smoothing additive regression using -splines, enabling a data-driven and expert-informed approximation of nonlinear terms. MiSSOC replaces complicating functions with additive spline surrogates and solves the surrogate MINLP—best when tackled with the -MINLP algorithm, followed by local NLP refinement—demonstrating strong performance on MINLPlib benchmarks and a Hydro Unit Commitment case study. The results show that MiSSOC yields high-quality solutions with significantly lower computational effort than solving the original problems directly, and can even surpass the best known feasible solutions in real-world instances. This work highlights the practical viability of combining data-driven surrogates with shape and bound constraints to tackle large, nonconvex MINLPs.

Abstract

Many real-world decision-making processes rely on solving mixed-integer nonlinear programming (MINLP) problems. However, finding high-quality solutions to MINLPs is often computationally demanding. This has motivated the development of specialized algorithms that take advantage of the structure of some MINLPs to improve their tractability. In this work, we propose the Mixed-integer Smoothing Surrogate Optimization with Constraints (MiSSOC) approach, a novel optimization algorithm that builds and solves surrogate problems for complex MINLPs which are more tractable in practice. MiSSOC integrates statistical modeling into mathematical optimization by approximating complex functions in an MINLP using smooth additive regression models with splines. Expert knowledge can be incorporated into the building phase of the approximating functions through shape constraints related to sign (or bounds), monotonicity and convexity over the observed domain. Thus, MiSSOC fills a gap in the literature by building surrogates that are both data-driven and knowledge-driven. The surrogate problem is formulated by replacing the original complex functions with their approximations. The proposed MiSSOC algorithm is evaluated through a set of experiments, including benchmark instances and a real-world case study. MiSSOC is tested with different state-of-the-art solvers and with the tailored Sequential Convex MINLP (SC-MINLP) algorithm. The latter exploits the separable structure of the surrogate functions, which results from the approximating functions being sums of piecewise univariate polynomials. MiSSOC is benchmarked against solving the original MINLPs directly. The results show that MiSSOC is an effective approach for addressing complex MINLPs through surrogate modeling, particularly when used in combination with the SC-MINLP algorithm.
Paper Structure (14 sections, 31 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 14 sections, 31 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary concepts
  3. Smooth univariate regression models
  4. Shape-constrained smooth univariate regression model
  5. Smooth additive regression models
  6. Smooth additive regression model with constraints
  7. Mixed-integer Smoothing Surrogate Optimization with Constraints (MiSSOC)
  8. The surrogate problem
  9. The surrogate formulation
  10. The overall algorithm
  11. Computational results
  12. Benchmark MINLPLib instances
  13. Real case study: Hydro Unit Commitment problem
  14. Conclusions and Perspectives

Figures (4)

  • Figure 1: $B-$spline functions of degrees 1, 2 and 3. The positions of the knots are indicated with vertical dotted lines.
  • Figure 2: Example of $B-$spline bases and knot sequences.
  • Figure 3: Example of function $f$ estimation (dashed line) from observed data (dots) using the $B-$spline basis in Figure \ref{['fig:preliminaries:Bspline_basis_extended']}. The $B-$spline functions are scaled by the regression coefficients, estimated by solving problem \ref{['eq:preliminaries:LSE_matrix_penalty']}.
  • Figure 4: Power generation surface $P(v, q)$ in the Hydro Unit Commitment problem and its bounded approximations using degrees $1$ to $7$ in both covariates $q$ and $v$, with $10$ intervals per variable.