From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task

TL;DR

This work presents a novel approach that leverages the 2D diffusion model and lateral continuity within the volume to enhance 3D volume electron microscopy (vEM) super-resolution and demonstrates the robustness and practical applicability of this framework for 3D volume super-resolution.

Abstract

Diffusion models have recently emerged as a powerful technique in image generation, especially for image super-resolution tasks. While 2D diffusion models significantly enhance the resolution of individual images, existing diffusion-based methods for 3D volume super-resolution often struggle with structure discontinuities in axial direction and high sampling costs. In this work, we present a novel approach that leverages the 2D diffusion model and lateral continuity within the volume to enhance 3D volume electron microscopy (vEM) super-resolution. We first simulate lateral degradation with slices in the XY plane and train a 2D diffusion model to learn how to restore the degraded slices. The model is then applied slice-by-slice in the lateral direction of low-resolution volume, recovering slices while preserving inherent lateral continuity. Following this, a high-frequency-aware 3D super-resolution network is trained on the recovery lateral slice sequences to learn spatial feature transformation across slices. Finally, the network is applied to infer high-resolution volumes in the axial direction, enabling 3D super-resolution. We validate our approach through comprehensive evaluations, including image similarity assessments, resolution analysis, and performance on downstream tasks. Our results on two publicly available focused ion beam scanning electron microscopy (FIB-SEM) datasets demonstrate the robustness and practical applicability of our framework for 3D volume super-resolution.

Figures (10)

• Figure 1: Super-Resolution of low-resolution vEM volume. Biological samples exhibit a consistent data distribution across spatial dimensions. In this work, we propose a training framework named Diffusion to Resolution (D2R) that leverages this intrinsic property to train 3D super-resolution networks without any high-resolution volumes as supervision. Our proposed 3D super-resolution network trained under D2R framework successfully performs 8x super-resolution on low-resolution volumes, achieving performance comparable to that of supervised training.
• Figure 2: An overview of the proposed D2R training framework and the DGEAN architecture. In Stage I, a 2D diffusion model is trained to restore high-resolution slices from degraded inputs. In Stage II, the trained model recovers the full volume in both XZ and YZ directions. This recovered volume serves as training data for the 3D convolution network in Stage III. After training in Stage III, the 3D convolution network performs inference on the XY plane to recover the high-resolution volume. The DGEAN network, a 3D convolutional network designed for volume super-resolution, shows excellent performance both with high-resolution volumes as supervision and with the D2R training framework, which operates without any high-resolution volume as supervision.
• Figure : Ground Truth
• Figure : Ground Truth
• Figure : Bicubic