MoSH: Modeling Multi-Objective Tradeoffs with Soft and Hard Bounds

TL;DR

MoSH introduces soft-hard functions (SHFs) to encode per-objective soft and hard bounds, enabling DM-aligned trade-off exploration in multi-objective optimization. It formulates a minimax objective over SHFs and proposes a two-step pipeline: MoSH-Dense densely samples the Pareto frontier via Bayesian optimization with random scalarizations, then MoSH-Sparse sparsifies the dense set with robust submodular optimization (SATURATE) to yield a compact, diverse PO set with theoretical guarantees. The approach is validated across diverse domains, including cervical cancer brachytherapy, engineering design, LLM personalization, and DL model selection, achieving superior SHF-defined utility and enabling DM satisfaction within a small, navigable set of options. The combination of SHFs, Bayesian sampling, and robust submodular sparsification demonstrates both strong theoretical guarantees and practical performance, offering a flexible framework for decision-makers facing expensive, multi-objective trade-offs. The work highlights significant implications for real-world optimization where practitioners inherently operate with soft and hard preferences.

Abstract

Countless science and engineering applications in multi-objective optimization (MOO) necessitate that decision-makers (DMs) select a Pareto-optimal solution which aligns with their preferences. Evaluating individual solutions is often expensive, necessitating cost-sensitive optimization techniques. Due to competing objectives, the space of trade-offs is also expansive -- thus, examining the full Pareto frontier may prove overwhelming to a DM. Such real-world settings generally have loosely-defined and context-specific desirable regions for each objective function that can aid in constraining the search over the Pareto frontier. We introduce a novel conceptual framework that operationalizes these priors using soft-hard functions, SHFs, which allow for the DM to intuitively impose soft and hard bounds on each objective -- which has been lacking in previous MOO frameworks. Leveraging a novel minimax formulation for Pareto frontier sampling, we propose a two-step process for obtaining a compact set of Pareto-optimal points which respect the user-defined soft and hard bounds: (1) densely sample the Pareto frontier using Bayesian optimization, and (2) sparsify the selected set to surface to the user, using robust submodular function optimization. We prove that (2) obtains the optimal compact Pareto-optimal set of points from (1). We further show that many practical problems fit within the SHF framework and provide extensive empirical validation on diverse domains, including brachytherapy, engineering design, and large language model personalization. Specifically, for brachytherapy, our approach returns a compact set of points with over 3% greater SHF-defined utility than the next best approach. Among the other diverse experiments, our approach consistently leads in utility, allowing the DM to reach >99% of their maximum possible desired utility within validation of 5 points.

Figures (21)

• Figure 1: Overall pipeline for our proposed method, MoSH. We evaluated MoSH on a real clinical case for cervical cancer brachytherapy treatment planning, where the objectives are to balance between the radiation dosage levels to the cancer tumor and to the nearby healthy organs -- bladder, rectum, and bowel. For the plots, we only showed two of the four dimensions in the multi-criterion objective for this task. In step 1, the iterations convey the gradual manner in which the sampled points move towards the clinician's region of ideal plans. In step 2, the Pareto dominant plan's metrics surpass those of the plan from an expert clinician in all dimensions, on real data.
• Figure 2: Example of a normalized soft-hard bounded utility function $u$ for the two-dimensional Branin-Currin function. The dashed vertical bars highlight the regions where the normalized values correspond to the hard, soft, and saturation point regions. The utility value associated with points below the hard bound is shown as $-2$ for illustration only. Computationally, we use $-\inf$.
• Figure 3: Complete-Mid configuration for the Branin-Currin synthetic two-objective function. Results are plotted using the metrics defined in Section \ref{['performance-criteria']}. The mean $\pm$ std. were computed over 6 independent runs.
• Figure 4: Top row: Four Bar Truss, narrow-mid. Bottom row: LLM personalization problem. Plots show the metrics defined in Section \ref{['performance-criteria']}. The mean $\pm$ std. were computed over 6 independent runs.
• Figure 5: Top row: cervical cancer brachytherapy treatment planning. Bottom row: deep learning model selection, which aims to select a fast and accurate neural network. The plots illustrate the metrics defined in Section \ref{['performance-criteria']}. The mean $\pm$ std. were computed over 6 independent runs.

Theorems & Definitions (25)

• Definition 1: Instantaneous SHF Regret
• Definition 2: Cumulative SHF Regret
• Definition 3: Bayes SHF Regret and Utility Ratio
• Theorem 4
• Definition 5: Submodular
• Lemma 6
• Theorem 7
• Theorem 8
• Definition 9: Pareto dominant
• Theorem 10