A Large Language Model and Denoising Diffusion Framework for Targeted Design of Microstructures with Commands in Natural Language

TL;DR

This work addresses the barrier to cross-disciplinary microstructure design by integrating NLP-driven natural language commands with conditional denoising diffusion probabilistic models (DDPMs) to generate microstructures with targeted properties. A modular pipeline combines a pretrained LLM for contextual data augmentation, a retrained NER to extract microstructure descriptors, a DDPM conditioned on embedded descriptor context, and a surrogate-filter system to rank outputs. Key contributions include contextual data augmentation for mechanics, a robust NER scheme with new entity labels, validation across multiple conditional tasks, and demonstration of end-to-end generation from text prompts. The framework enables accessible inverse design of hyperelastic microstructures starting from intuitive natural language inputs, with potential for adaptation to diverse material systems and design objectives.

Abstract

Microstructure plays a critical role in determining the macroscopic properties of materials, with applications spanning alloy design, MEMS devices, and tissue engineering, among many others. Computational frameworks have been developed to capture the complex relationship between microstructure and material behavior. However, despite these advancements, the steep learning curve associated with domain-specific knowledge and complex algorithms restricts the broader application of these tools. To lower this barrier, we propose a framework that integrates Natural Language Processing (NLP), Large Language Models (LLMs), and Denoising Diffusion Probabilistic Models (DDPMs) to enable microstructure design using intuitive natural language commands. Our framework employs contextual data augmentation, driven by a pretrained LLM, to generate and expand a diverse dataset of microstructure descriptors. A retrained NER model extracts relevant microstructure descriptors from user-provided natural language inputs, which are then used by the DDPM to generate microstructures with targeted mechanical properties and topological features. The NLP and DDPM components of the framework are modular, allowing for separate training and validation, which ensures flexibility in adapting the framework to different datasets and use cases. A surrogate model system is employed to rank and filter generated samples based on their alignment with target properties. Demonstrated on a database of nonlinear hyperelastic microstructures, this framework serves as a prototype for accessible inverse design of microstructures, starting from intuitive natural language commands.

Figures (15)

• Figure 1: Schematic of overall framework components. The NLP component framework (top) involves a large language model for the generation and augmentation of text descriptor database and a retrained named entity recognition model to extract microstructure descriptors from text. The microstructure generation component (bottom) involves a denoising diffusion probabilistic model framework and system of surrogate models for ranking and filtering the massively generate microstructure samples.
• Figure 2: A sample of the training dataset of the framework consists of the FEM recorded homogenized energy functional under tension as well as the topology label, the inclusion ratio, a symbolic expression of the energy functional, and a text description of the microstructure.
• Figure 3: Embedding of two semantically identical microstructure descriptions with different formatting of descriptors leads to different text embeddings.
• Figure 4: (a) Results for one prompt before retraining the NER model. (b) NER results for 5 prompts and contexts from the Mechanical MNIST testing set after retraining.
• Figure 5: Overall performance of all the 219 combination prompts for the seven tests described in Table \ref{['tab:tests_contexts']}.