Learning ORDER-Aware Multimodal Representations for Composite Materials Design

TL;DR

ORDER addresses the challenge of designing composites with continuous, high-dimensional fiber distributions by learning ordinal-aware, multimodal representations that align microstructure images with tabular descriptors. It combines cross-modal alignment with intra-modal ordinal learning, enhanced by preference-guided multitask optimization and LoRA-based fine-tuning of a CLIP vision backbone to handle extreme data scarcity. Across retrieval, property prediction, and descriptor-conditioned microstructure generation, ORDER delivers consistent improvements over baselines and enables meaningful interpolation within unseen design regions. This framework offers a principled path to interpolate in infinite design spaces and can generalize to other domains requiring continuous, multimodal material representations.

Abstract

Artificial intelligence (AI) has shown remarkable success in materials discovery and property prediction, particularly for crystalline and polymer systems where material properties and structures are dominated by discrete graph representations. Such graph-central paradigm breaks down on composite materials, which possess continuous and nonlinear design spaces that lack well-defined graph structures. General composite descriptors, e.g., fiber volume and misalignment angle, cannot fully capture the fiber distributions that fundamentally determine microstructural characteristics, necessitating the integration of heterogeneous data sources through multimodal learning. Existing alignment-oriented multimodal frameworks have proven effective on abundant crystal or polymer data under discrete, unique graph-property mapping assumptions, but fail to address the highly continuous composite design space under extreme data scarcity. In this work, we introduce ORDinal-aware imagE-tabulaR alignment (ORDER), a multimodal pretraining framework that establishes ordinality as a core principle for composite material representations. ORDER ensures that materials with similar target properties occupy nearby regions in the latent space, which effectively preserves the continuous nature of composite properties and enables meaningful interpolation between sparsely observed designs. We evaluate ORDER on a public Nanofiber-enforced composite dataset and an internally curated dataset that simulates the construction of carbon fiber T700 with diverse fiber distributions. ORDER achieves consistent improvements over state-of-the-art multimodal baselines across property prediction, cross-modal retrieval, and microstructure generation tasks.

Figures (5)

• Figure 1: a The pretraining pipeline of ORDER. Step 1, raw composite data images and target properties are obtained by simulating on various descriptors, and organized as pairs. Step 2, paired tabular descriptors and microstructure images are encoded into a shared latent space via dedicated encoders. Step 3, we apply cross-modal contrastive learning between modalities to enforce image-tabular alignment, and ordinal-aware contrastive learning within each modality to produce property-ordered embeddings. Preference-guided multitask optimization addresses potential conflicts during the bi-objective optimization. Step 4, the training process yields an aligned and ordinal-aware multimodal latent space. b Downstream tasks. The pretrained features are then frozen and serve as the initial starting point for various downstream tasks: (i) Cross-modal retrieval by finding the most similar candidate features with the query feature. The ordinal awareness encoded in features ensures physically meaningful candidates. (ii) Property prediction by training lightweight prediction heads based on the pre-aligned features. (iii) Descriptor-conditioned microstructure generation from tabular inputs. Based on the pretrained feature space, a prior network is trained to translate tabular features into image features and a decoder network learns to reverse them into images.
• Figure 2: a Cross-modal retrieval results w.r.t. accuracy with varying number of retrieved candidates ($k$). ORDER variants consistently outperform vanilla cross-modal contrastive learning (CMCL) and MatMCL. b Examples on top-5 retrieved images given tabular descriptors. The left panel shows query descriptors and the corresponding ground-truth image. The middle panel shows top-5 retrieved examples, with their target property values displayed beneath each retrieved item. Samples with green borders represent correct retrievals (ground truth), while red borders indicate unwanted candidates with substantially different target properties. The right panel presents statistics of the properties of retrieved samples. ORDER achieves markedly lower property deviation errors across retrieved candidates. c Examples on top-5 retrieved descriptors given a query image.
• Figure 3: Target property prediction performance on Composite and Nanofiber datasets. For multimodal pretraining methods (ORDER, MatMCL, CMCL), extracted features are frozen and used to train an MLP for property prediction. All multimodal pretraining methods use ViT-B/16 as backbone. ORDER achieves substantial improvements on both datasets compared with multimodal and modality-specific baselines. The modality-fusion results (highlighted with borders and shadows) bring even better performances by incorporating both modality strengths.
• Figure 4: Descriptor-conditioned microstructure generation. a Representative generation examples from ORDER-dyn, CMCL, and MatMCL. Two randomly selected samples are shown for each method. In-distribution samples were observed during prior and decoder training, while out-of-distribution samples are unseen. These examples provide qualitative assessments of pretrained features when assisting the generation of microstructures. b Quantitative evaluation of generated samples using five complementary metrics: FID, KID, LPIPS, IS, and PSNR. Outer rings in radar plots indicate better performance. ORDER-dyn achieves consistent improvements compared with multimodal baselines.
• Figure 5: Visualization of the pretrained multimodal representations using target property 'Elongation'. a The t-SNE projection of multimodal feature space. Darker colors correspond to higher target property values. 'Initial' figures show features before pretraining, where image and tabular features distribute apart. After pretraining, ORDER achieves both cross-modal alignment (overlapping distributions) and property-based ordering (color gradients). For Nanofiber data with multiple target properties, features exhibit non-linear low-to-high property trends (left to right). b Cross-modal feature similarity matrices for samples sorted by target property values. Brighter color indicates higher similarity and alignment. ORDER exhibits alignment not only between cross-modal pairs, but also with samples with proximate property values.