MECHBench: A Set of Black-Box Optimization Benchmarks originated from Structural Mechanics

TL;DR

MECHBench provides a realistic, application-oriented benchmark suite for black-box optimization in structural mechanics by grounding three crashworthiness problems in modular, OpenRadioss-driven simulations. It introduces a standardized workflow and Python interface that decouples optimization from simulation, enabling fair comparisons of gradient-free approaches across problems with tunable dimensionality and explicit objectives such as SEA, mass, and load uniformity LU. Each problem includes both a standard constrained formulation and a reformulated unconstrained objective to support different optimizer styles, along with detailed problem descriptions and material settings to improve reproducibility. The work contributes open-source code, data, and guidelines to foster reproducibility, scalability, and cross-community adoption, and plans to expand with transformations, surrogates, and multi-fidelity evaluations to better reflect real engineering design pipelines.

Abstract

Benchmarking is essential for developing and evaluating black-box optimization algorithms, providing a structured means to analyze their search behavior. Its effectiveness relies on carefully selected problem sets used for evaluation. To date, most established benchmark suites for black-box optimization consist of abstract or synthetic problems that only partially capture the complexities of real-world engineering applications, thereby severely limiting the insights that can be gained for application-oriented optimization scenarios and reducing their practical impact. To close this gap, we propose a new benchmarking suite that addresses it by presenting a curated set of optimization benchmarks rooted in structural mechanics. The current implemented benchmarks are derived from vehicle crashworthiness scenarios, which inherently require the use of gradient-free algorithms due to the non-smooth, highly non-linear nature of the underlying models. Within this paper, the reader will find descriptions of the physical context of each case, the corresponding optimization problem formulations, and clear guidelines on how to employ the suite.

Figures (13)

• Figure 1: Proposed optimization loop. To initiate the loop, the optimization algorithm, objective values, and problem number must be defined a priori. The blocks connected with solid arrows represent the necessary steps of the loop, whereas the dashed arrows indicate optional steps or processes that do not interrupt the loop. Adapted from hunkeler_shape_2013Volz2007.
• Figure 2: MECHBench directory structure. Currently, the project contains 3 levels; Level 1 is where the main.py should go. On the same level, the repository has the folder results which contain sample simulation-generated files. The main source code is located under src/, where the sob/subpackage holds the core functionality. This includes modules for problem definitions (problems.py), finite element routines (fem.py), mesh handling (mesh.py), and solvers (solver.py). Within the second level or sob/, there are two utility subdirectories. The utils/ folder provides helper scripts such as run_openradioss.py for starting the subprocesses and solver_setup.py for preparing solver configurations. Finally, the third level or lib/ folder contains geometry and mesh generation scripts, including gmsh_base_meshes.py as a foundation, and specialized generators like starbox_gmsh.py, crashtube_gmsh.py, and three_point_bending_gmsh.py.
• Figure 3: Star-shaped crash-box case. The structure is subdivided into smaller connected parts, forming a finite element mesh shown by the blue lines. In the mm$z$-direction, the mesh is arranged in 30 rows of elements, wherein each row is assigned the same thickness value $t_h$.
• Figure 4: Optimization problem parameterization for $d\leq5$. The ranges of each variable are expressed in mm. The drawings are not in scale. For the cases where thickness is not controlled such as the cases where $d=\{1,2,4\}$, the wall thickness $t_h$ is set constant to 2.1 mm.
• Figure 5: Thickness distribution example for $d=7$, wherein 3 control points are set at uniformly distributed heights between $0$ and $120$mm and each point has an assigned $t_h(z)$. The wall thickness $\bar{t_h}$ at any intermediate point $\bar{z}$ is computed via linear interpolation of assigned $t_{h}$ of the neighboring control points. This procedure is used to compute the wall thickness $t_h$ of a row of finite elements by getting the interpolated $\bar{t_h}$ at the element barycenter.