Generative Multimodal Multiscale Data Fusion for Digital Twins in Aerosol Jet Electronics Printing

TL;DR

The paper tackles the challenge of modeling and fusing multimodal, multiscale PSP data in Aerosol Jet Printing to support digital twins. It introduces a diffusion-model-based framework that first registers AJP PSP features from OM and CP data and then performs a denoising diffusion fusion to infer PSP causality and synthesize fused features, with hyperparameters finely tuned using SSIM. Through a case study on OM and CP data from a 16-hour print, the approach demonstrates the ability to capture complex PSP relationships and generate novel features, enabling deeper insight into dynamic manufacturing. The work advances predictive control, quality assurance, and DT construction in electronics AM by providing data-driven generative simulations and highlighting potential for real-time monitoring and domain-knowledge-enhanced fusion. The methodology, including DDIM-based fusion and targeted data registration, offers a path toward more accurate, interpretable, and actionable digital twins in AJP-based manufacturing.

Abstract

The rising demand for high-value electronics necessitates advanced manufacturing techniques capable of meeting stringent specifications for precise, complex, and compact devices, driving the shift toward innovative additive manufacturing (AM) solutions. Aerosol Jet Printing (AJP) is a versatile AM technique that utilizes aerosolized functional materials to accurately print intricate patterns onto diverse substrates. Machine learning (ML)- based Process-Structure-Property (PSP) modeling is essential for enhancing AJP manufacturing, as it quantitatively connects process parameters, structural features, and resulting material properties. However, current ML approaches for modeling PSP relationships in AJP face significant limitations in handling multimodal and multiscale data, underscoring a critical need for generative methods capable of comprehensive analysis through multimodal and multiscale fusion. To address this challenge, this study introduces a novel generative modeling methodology leveraging diffusion models for PSP data fusion in AJP. The proposed method integrates multimodal, multiscale PSP features in two phases: (1) registering the features, and (2) fusing them to generate causal relationships between PSP attributes. A case study demonstrates the registration and fusion of optical microscopy (OM) images and confocal profilometry (CP) data from AJP, along with the fine-tuning of the fusion step. The results effectively capture complex PSP relationships, offering deeper insights into digital twins of dynamic manufacturing systems.

Figures (7)

• Figure 1: The Schematic Diagram of the Key Stages in the AJP Process and the Real Example of Optical Monitoring Images of Printed Lines in Aerosol Jet Printing. (A) This subfigure illustrates the five main stages involved in the AJP process: aerosol droplet generation (atomization), aerosol transport, aerosol beam collimation, aerodynamic focusing, and impaction; (B) This subfigure shows the examples of the post-print optical monitoring images of printed lines at different time points during the AJP process. Each of the four sample images contains four lines, printed sequentially from top to bottom. The top line in each image was printed earlier, while the subsequent lines show gradual increases in width, indicating variations in line morphology over time despite consistent process parameters yoo2022optical.
• Figure 2: An Overall Framework for Multimodal Multiscale AJP PSP Fusion.
• Figure 3: Example of Registered and Extracted $\text{ROI}_{\text{OM}}$ and $\text{ROI}_{\text{CP}}$Images for Data Fusion. This figure shows an example of registered and extracted $\text{ROI}_{\text{OM}}$ and $\text{ROI}_{\text{CP}}$images, demonstrating both spatiotemporal alignments achieved during the data registration process. This alignment ensures consistency between the two data types helps to form a reliable basis for the subsequent data fusion process.
• Figure 4: A Denoising Diffusion-Based Fusion of $\text{ROI}_{\text{OM}}$ and $\text{ROI}_{\text{CP}}$
• Figure 5: Fine-Tuning Results for Hyperparameters $\eta$ and $\psi$.