Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior

Kyungryun Lee, Won-Ki Jeong

TL;DR

This work tackles axial super-resolution for anisotropic 3D microscopy without requiring isotropic ground-truth volumes. It introduces an implicit neural representation (INR) whose parameters are optimized under a 2D diffusion prior learned from high-resolution XY slices using Score Distillation Sampling (SDS), ensuring 3D coherence across ZX/ZY planes. The approach is evaluated on simulated FIB-25 data and real CREMI and Zebrafish retina volumes, showing superior PSNR and SSIM with reduced artifacts and misalignment compared to diffusion-model baselines, while maintaining fidelity to low-resolution measurements. By avoiding direct 3D generation and enforcing coherence through the INR-Diffusion Prior framework, the method provides reliable, detailed, and scalable axial super-resolution suitable for practical 3D microscopy workflows. Code is released on GitHub, facilitating adoption and further development.

Abstract

Analysis and visualization of 3D microscopy images pose challenges due to anisotropic axial resolution, demanding volumetric super-resolution along the axial direction. While training a learning-based 3D super-resolution model seems to be a straightforward solution, it requires ground truth isotropic volumes and suffers from the curse of dimensionality. Therefore, existing methods utilize 2D neural networks to reconstruct each axial slice, eventually piecing together the entire volume. However, reconstructing each slice in the pixel domain fails to give consistent reconstruction in all directions leading to misalignment artifacts. In this work, we present a reconstruction framework based on implicit neural representation (INR), which allows 3D coherency even when optimized by independent axial slices in a batch-wise manner. Our method optimizes a continuous volumetric representation from low-resolution axial slices, using a 2D diffusion prior trained on high-resolution lateral slices without requiring isotropic volumes. Through experiments on real and synthetic anisotropic microscopy images, we demonstrate that our method surpasses other state-of-the-art reconstruction methods. The source code is available on GitHub: https://github.com/hvcl/INR-diffusion.

Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior

TL;DR

This work tackles axial super-resolution for anisotropic 3D microscopy without requiring isotropic ground-truth volumes. It introduces an implicit neural representation (INR) whose parameters are optimized under a 2D diffusion prior learned from high-resolution XY slices using Score Distillation Sampling (SDS), ensuring 3D coherence across ZX/ZY planes. The approach is evaluated on simulated FIB-25 data and real CREMI and Zebrafish retina volumes, showing superior PSNR and SSIM with reduced artifacts and misalignment compared to diffusion-model baselines, while maintaining fidelity to low-resolution measurements. By avoiding direct 3D generation and enforcing coherence through the INR-Diffusion Prior framework, the method provides reliable, detailed, and scalable axial super-resolution suitable for practical 3D microscopy workflows. Code is released on GitHub, facilitating adoption and further development.

Abstract

Analysis and visualization of 3D microscopy images pose challenges due to anisotropic axial resolution, demanding volumetric super-resolution along the axial direction. While training a learning-based 3D super-resolution model seems to be a straightforward solution, it requires ground truth isotropic volumes and suffers from the curse of dimensionality. Therefore, existing methods utilize 2D neural networks to reconstruct each axial slice, eventually piecing together the entire volume. However, reconstructing each slice in the pixel domain fails to give consistent reconstruction in all directions leading to misalignment artifacts. In this work, we present a reconstruction framework based on implicit neural representation (INR), which allows 3D coherency even when optimized by independent axial slices in a batch-wise manner. Our method optimizes a continuous volumetric representation from low-resolution axial slices, using a 2D diffusion prior trained on high-resolution lateral slices without requiring isotropic volumes. Through experiments on real and synthetic anisotropic microscopy images, we demonstrate that our method surpasses other state-of-the-art reconstruction methods. The source code is available on GitHub: https://github.com/hvcl/INR-diffusion.
Paper Structure (11 sections, 6 equations, 3 figures, 1 table)

This paper contains 11 sections, 6 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Implicit neural representations for isotropic volume reconstruction
  4. Diffusion models and score distillation sampling
  5. Experiments
  6. Datasets and implementation details
  7. Simulation studies on FIB25
  8. Real world anisotropic volumes
  9. Conclusion and Future work
  10. Acknowledgements.
  11. Disclosure of Interests.

Figures (3)

  • Figure 1: (top): The overview of our framework. First, we train the diffusion model using XY slices of the anisotropic volume, where the Z resolution significantly degrades compared to the XY resolution. During the optimization process of the INR, we randomly sample batches of ZX,ZY planes and optimize the loss. (bottom): The evolution of applying the diffusion prior: queried image(first row), $t$-step noisy image(second row) and the scaled difference between the original image and the refined output of the diffusion model. The discrepancy between the image before and after passing the diffusion model provides a guiding direction.
  • Figure 2: Visual comparison of reconstructing the simulated FIB25 volume with a downsampling factor of 8. The volume is viewed in the ZY direction. All methods except ours show disconnection in the intersection.
  • Figure 3: (row1): Visual comparsion of reconstructing real anisotropic electron microscopy image with a downsampling factor of 10. Inside the red circle shows an example of hallucination. As there is no ground truth image, we use the lateral slices(XY) as a reference. (row2): A merged fluorescence microscopy image of a zebrafish retina cell composed of two channels: the red channel representing the nuclei and the green channel representing the nuclear envelope.