Multi-Objective Optimization and Hyperparameter Tuning With Desirability Functions

TL;DR

This work presents spotdesirability, a Python implementation of the desirability-function framework for multi-objective optimization, and demonstrates its use across classical optimization, surrogate-model-based optimization, and hyperparameter tuning. It translates multiple objectives into individual desirabilities and aggregates them with a geometric mean, enabling a unified objective $D = \left(\prod_{r=1}^R d_r\right)^{1/R}$ that guides optimization. The paper showcases both single- and multi-objective tuning strategies, compares direct Nelder-Mead optimization with surrogate- and desirability-based approaches, and introduces space-filling metrics via the Morris-Mitchell criterion to balance exploration and exploitation. Practical contributions include a Python ecosystem (spotdesirability, spotoptim) with concrete workflows for predictive modeling, neural-network hyperparameter tuning, and Pareto-front visualization, facilitating reproducible, multi-objective optimization in real-world settings. The results indicate that desirability-based methods can achieve competitive trade-offs, improve interpretability of objectives, and provide flexible tooling for hyperparameter tuning and design-space exploration, with formal handling of missing values, tolerances, and non-standard desirability shapes.

Abstract

The desirability-function approach is a widely adopted method for optimizing multiple-response processes. Kuhn (2016) implemented the packages desirability and desirability2 in the statistical programming language R, but no comparable packages exists for Python. The goal of this article is to provide an introduction to the desirability function approach using the Python package spotdesirability, which is available as part of the sequential parameter optimization framework. After a brief introduction to the desirability function approach, three examples are given that demonstrate how to use the desirability functions for (i) classical optimization, (ii) surrogate-model based optimization, and (iii) hyperparameter tuning. An extended Morris-Mitchell criterion, which allows the computation of the search-space coverage, is proposed and used in a fourth example to handle the exploration-exploitation trade-off in optimization. Finally, infill-diagnostic plots are introduced as a tool to visualize the locations of the infill points with respect to already existing points.

Figures (37)

• Figure 1: Examples of the three primary desirability functions. Panel (a) shows an example of a larger--is--better function, panel (b) shows a smaller--is--better desirability function and panel (c) shows a function where the optimal value corresponds to a target value. Not that increasing the scale parameter makes it more difficult to achieve higher desirability, while values smaller than 1 make it easier to achieve good results.
• Figure 2: An example of a desirability function created using the DArb function. The desirability function is a logistic curve that is defined by 20 points on the range [-5, 5].
• Figure 3: An example of a box-like desirability function that assigns zero desirability to values outside of the range [-1.682, 1.682].
• Figure 4: Desirability function for categorical values. The desirability values are assigned to three categories: 'value1', 'value2', and 'value3'.
• Figure 5: Central composite design (CCD) for the chemical reaction example

Theorems & Definitions (7)

• Example 2.1: Logistic Desirability Function
• Example 2.2: Box-Constraint Desirability Function
• Example 2.3: Desirability Function for Categorical Inputs
• Example 4.1: Computing Desirability at the Center Point
• Example 6.1: Testing the Desirability Function
• Example 8.1: Usage of mmphi_intensive
• Example 8.2: Computation of the Improvement