Deep Learning based Optical Image Super-Resolution via Generative Diffusion Models for Layerwise in-situ LPBF Monitoring

TL;DR

Generative deep learning models are implemented to link low-cost, low-resolution images of the build plate to detailed high-resolution optical images of the build plate, enabling cost-efficient process monitoring and designing a framework based upon the Segment Anything foundation model to recreate the 3D morphology of the printed part and analyze the surface roughness of the reconstructed samples.

Abstract

The stochastic formation of defects during Laser Powder Bed Fusion (L-PBF) negatively impacts its adoption for high-precision use cases. Optical monitoring techniques can be used to identify defects based on layer-wise imaging, but these methods are difficult to scale to high resolutions due to cost and memory constraints. Therefore, we implement generative deep learning models to link low-cost, low-resolution images of the build plate to detailed high-resolution optical images of the build plate, enabling cost-efficient process monitoring. To do so, a conditional latent probabilistic diffusion model is trained to produce realistic high-resolution images of the build plate from low-resolution webcam images, recovering the distribution of small-scale features and surface roughness. We first evaluate the performance of the model by analyzing the reconstruction quality of the generated images using peak-signal-to-noise-ratio (PSNR), structural similarity index measure (SSIM) and wavelet covariance metrics that describe the preservation of high-frequency information. Additionally, we design a framework based upon the Segment Anything foundation model to recreate the 3D morphology of the printed part and analyze the surface roughness of the reconstructed samples. Finally, we explore the zero-shot generalization capabilities of the implemented framework to other part geometries by creating synthetic low-resolution data.

Figures (15)

• Figure 1: a) A schematic of the proposed workflow. During the build process, layerwise high-resolution optical images and low-resolution webcam images are collected. The low resolution images undergo image translation to the space of high-resolution optical images via a generative model. The performance of the upscaling process is evaluated based on distributional comparisons between the high-resolution ground truth and the generated high-resolution images. b) A diffusion model is used to generate new high-resolution samples conditioned on the input low-resolution webcam images, through an iterative denoising training process.
• Figure 2: a), b) Two autoencoder networks are trained to encode the layerwise patches for each patch-wise image in the dataset to a latent space, $z$. One encoder is trained to learn an embedding of the high-resolution (HR) data, and a second encoder is trained to learn an embedding of the low-resolution (LR) input data. c) During the diffusion model training process, the trained autoencoders are used to first project the low-resolution data into a compressed latent space. Next, a conditional diffusion model generates an appropriate high-resolution latent vector from the low-resolution input data. The high-resolution decoder network is then used to reconstruct a predicted high-resolution sample.
• Figure 3: Layer-wise images are collected during the build process for two sample groups of parts. These images are collected in both high resolution with a Basler AC45472-17um camera and low resolution with a 1080p Logitech StreamCam camera. a)$n$ = 65 layer-wise images are collected for the first build. b)$n$ = 115 layer-wise images are collected for the second build.
• Figure 4: A two-stage process is used to automatically segment arbitrary patches of the build plate with the pre-trained Segment Anything foundation model. In the first stage, an adaptive threshold is applied to extract areas of the part surface, which are labeled to indicate the visible part components. In the second stage, query points are sampled for use with Segment-Anything, which provides a series of part masks. These part masks are aggregated to form the final composite mask.
• Figure 5: Layer-wise registration between the high-resolution and low-resolution images obtained at different orientations within the build chamber, the two image fidelities are registered to each other following an initial warping transformation. a) A comparison of the overlaid part samples after an initial warp to approximate the perspective transformation from the original point-of-view to a top-down view. The displacement between each part across the two image fidelities are calculated as $\Delta \mathbf{s}$. b) The measured displacements between the parts are interpolated across the build plate to form a deformation map, $-\Delta \mathbf{s}(x,y)$. c) The high-resolution image is transformed according to the deformation map $-\Delta \mathbf{s}$ to achieve pixel-wise agreement.