MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Models
Yash Patawari Jain, Daniele Grandi, Allin Groom, Brandon Cramer, Christopher McComb
TL;DR
Material selection in conceptual design suffers from the absence of standardized benchmarks for evaluating algorithmic approaches. The paper introduces MSEval, a dataset of expert evaluations spanning diverse design briefs and criteria, intended as a gold standard for benchmarking material-selection methods. It outlines how the dataset can be used to assess performance, generalization, and alignment with expert judgments, and suggests workflows for integrating the data into model evaluation. By providing a practical, consistent resource, the work aims to accelerate research in material selection and enable more meaningful cross-method comparisons.
Abstract
Material selection plays a pivotal role in many industries, from manufacturing to construction. Material selection is usually carried out after several cycles of conceptual design, during which designers iteratively refine the design solution and the intended manufacturing approach. In design research, material selection is typically treated as an optimization problem with a single correct answer. Moreover, it is also often restricted to specific types of objects or design functions, which can make the selection process computationally expensive and time-consuming. In this paper, we introduce MSEval, a novel dataset which is comprised of expert material evaluations across a variety of design briefs and criteria. This data is designed to serve as a benchmark to facilitate the evaluation and modification of machine learning models in the context of material selection for conceptual design.