MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections

TL;DR

This work pretrain an INR model to transform 2D axially-projected images into a pre-liminary 3D volume, which enriches the diffusion process with structured 3D information, enhancing detail and reducing noise in localized 2D images.

Abstract

Volumetric optical microscopy using non-diffracting beams enables rapid imaging of 3D volumes by projecting them axially to 2D images but lacks crucial depth information. Addressing this, we introduce MicroDiffusion, a pioneering tool facilitating high-quality, depth-resolved 3D volume reconstruction from limited 2D projections. While existing Implicit Neural Representation (INR) models often yield incomplete outputs and Denoising Diffusion Probabilistic Models (DDPM) excel at capturing details, our method integrates INR's structural coherence with DDPM's fine-detail enhancement capabilities. We pretrain an INR model to transform 2D axially-projected images into a preliminary 3D volume. This pretrained INR acts as a global prior guiding DDPM's generative process through a linear interpolation between INR outputs and noise inputs. This strategy enriches the diffusion process with structured 3D information, enhancing detail and reducing noise in localized 2D images. By conditioning the diffusion model on the closest 2D projection, MicroDiffusion substantially enhances fidelity in resulting 3D reconstructions, surpassing INR and standard DDPM outputs with unparalleled image quality and structural fidelity. Our code and dataset are available at https://github.com/UCSC-VLAA/MicroDiffusion.

Figures (7)

• Figure 1: Background and concept of MicroDiffusion-enabled volumetric microscopy. (a) Conventional 3D laser scanning microscopy, while depth-resolvable due to its point-scanning 3D data acquisition scheme, suffers from slow imaging speed. (b) Volumetric microscopy using a non-diffracting laser beam provides fast volumetric imaging by axially projecting 3D volumes onto 2D images but lacks depth information within each acquired 2D image. (c) Our proposed MicroDiffusion model is employed as a digital backend for 3D volumetric reconstruction from 2D projections acquired in (b). MicroDiffusion significantly enhances volumetric imaging performance, providing a synergistic balance between imaging speeds and depth-resolving capabilities.
• Figure 2: Pipeline of MicroDiffusion. Step 1, we pre-train an INR which provides rough reconstructed images. Step 2, the 2D projections and 3D coordinates are used as the classifier-free guidance of the MicroDiffusion, and the INR output is integrated into the noisy image as guidance during the diffusion process. Detailed information is available at Sec. \ref{['sec:Method']}.
• Figure 3: Qualitative results: (a) Comparative visualization of slices from 3D reconstructions with different methods. Observable differences between the INR reconstruction, MicroDiffusion reconstruction, and ground truth are indicated with white arrows. (b) 3D vasculature. Scale bar: 30 $\mu$m.
• Figure 4: Performance metrics across different step lengths.
• Figure 5: Reconstruction of the depth-resolved sparsely distributed neuron images and depth-resolved volumetric projections with different step lengths.