ViT-Transformer: Self-attention mechanism based constitutive modeling for nonlinear heterogeneous materials

TL;DR

This work addresses the computational burden of multiscale simulations for nonlinear heterogeneous materials by introducing ViT-Transformer, a surrogate model that uses a Vision Transformer encoder to extract microstructural features from RVE images and a Transformer decoder to predict macroscopic stresses from strain histories. A random extract training strategy and data augmentation enable robust generalization to variable-length sequences and unseen microstructures/loading conditions, while data are generated from diverse RVEs and validated against high-fidelity homogenization results. The approach demonstrates high accuracy (e.g., ~1.16% relative error on test data) and outperforms GRU-based counterparts in long sequences, with strong generalization to unseen microstructures and loading protocols. This signifies a scalable, accurate pathway for rapid multiscale predictions in fiber-reinforced composites and similar heterogeneous materials.

Abstract

Multi-scale simulations of nonlinear heterogeneous materials and composites are challenging due to the prohibitive computational costs of high-fidelity simulations. Recently, machine learning (ML) based approaches have emerged as promising alternatives to traditional multiscale methods. However, existing ML surrogate constitutive models struggle in capturing long-range dependencies and generalization across microstructures. The recent advancements in attention-based Transformer architectures open the door to a more powerful class of surrogate models. Attention mechanism has demonstrated remarkable capabilities in natural language processing and computer vision. In this work, we introduce a surrogate (meta) model, namely ViT-Transformer, using a Vision Transformer (ViT) encoder and a Transformer-based decoder which are both driven by the self-attention mechanism. The ViT encoder extracts microstructural features from material images, while the decoder is a masked Transformer encoder that combines the latent geometrical features with the macroscopic strain input sequence to predict the corresponding stress response. To enhance training, we propose a random extract training algorithm that improves robustness to sequences of variable length. We design and construct a compact yet diverse dataset via data augmentation, and validate the surrogate model using various composite material images and loading scenarios. Several numerical examples are provided to show the effectiveness and accuracy of the ViT-Transformer model and the training algorithm.

Figures (16)

• Figure 1: An example RVE model generated by the preprocessing script with randomly distributed fibers. Fibers are located along the Z direction.
• Figure 2: Architecture of the original Transformer model in NIPS2017_3f5ee243.
• Figure 3: Architecture of the ViT-Transformer.
• Figure 4: An example of dataset augmentation: a) the original image, b) flip along the y-axis, c) rotate 180 degrees clockwise, d) flip along the x-axis, e) rotate 90 degrees clockwise, and f) rotate 270 degrees clockwise.
• Figure 5: Training and test loss curves versus epochs.