Self-supervised feature distillation and design of experiments for efficient training of micromechanical deep learning surrogates

TL;DR

The paper addresses the challenge of efficiently training micromechanical deep learning surrogates by coupling intelligent microstructure representation with principled experimental design. It introduces three feature descriptors (VAE latent, self-supervised contrastive statistics, and classical GSH-based descriptors) and evaluates three space-filling designs (maximin, maxPro, and data twinning) within a CNN/U-net surrogate framework trained on elastic micromechanical simulations. Across a parametric study, the contrastive self-supervised features with a maximin-type design (contrastive cMm) offer robust improvements, with up to about 8% reduction in validation loss for moderate data sizes and reasonable stability across settings; data twinning provides a steady ~5% boost. The results demonstrate that careful experimental design and meaningful microstructure statistics substantially enhance surrogate performance and are likely to become more beneficial as problem sizes grow, enabling scalable micromechanical surrogates and informing broader image-based design tasks.

Abstract

Machine learning surrogate emulators are needed in engineering design and optimization tasks to rapidly emulate computationally expensive physics-based models. In micromechanics problems the local full-field response variables are desired at microstructural length scales. While there has been a great deal of work on establishing architectures for these tasks there has been relatively little work on establishing microstructural experimental design strategies. This work demonstrates that intelligent selection of microstructural volume elements for subsequent physics simulations enables the establishment of more accurate surrogate models. There exist two key challenges towards establishing a suitable framework: (1) microstructural feature quantification and (2) establishment of a criteria which encourages construction of a diverse training data set. Three feature extraction strategies are used as well as three design criteria. A novel contrastive feature extraction approach is established for automated self-supervised extraction of microstructural summary statistics. Results indicate that for the problem considered up to a 8\% improvement in surrogate performance may be achieved using the proposed design and training strategy. Trends indicate this approach may be even more beneficial when scaled towards larger problems. These results demonstrate that the selection of an efficient experimental design is an important consideration when establishing machine learning based surrogate models.

Figures (17)

• Figure 1: Overall approach is to identify most unique and informative MVEs for subsequent physics evaluation and surrogate training. Hypothesis is that more efficient training may be performed if MVEs are chosen using an appropriate design criteria.
• Figure 2: VAE schematic for extracting localized MVE features.
• Figure 3: Self-supervised approach for 3D MVE feature extraction and network training. Subsampling of MVEs enables self-supervised learning and, critically, encourages the learning of statistical descriptors.
• Figure 4: Novelty of the feature extraction procedure is that it intrinsically operates on image statistics via construction of the network. Mean and variance of spatial-orientation feature maps are combined with volume averaged orientation features prior to passing through the final MLP. This encourages the network to separately construct orientation and spatial-orientation statistics prior to mixing in the final MLP.
• Figure 5: Maximum projection design criteria ensures good spreading in all possible subspace projections. This ensures that even when unknown unimportant features are present the resulting design still exhibits desirable space-filling properties in the effective lower dimensional space.