Step-Calibrated Diffusion for Biomedical Optical Image Restoration

TL;DR

Biomedical optical SRH imaging suffers from unpredictable degradation that jeopardizes diagnosis. RSCD addresses this with a step-calibrated diffusion framework that uses a learned $t_{pred}$ and dynamic recalibration to restore low-quality SRH images using unpaired high-quality data, while avoiding hallucinations. Across unpaired and near-registered evaluations, RSCD outperforms baselines on perceptual and pixel-level metrics and is preferred by experts, with demonstrated improvements in automated brain tumor diagnostics and deep-tissue z-stack restoration. These results indicate RSCD can enhance intraoperative imaging reliability and aid precision medicine by enabling safer, more accurate clinical decisions.

Abstract

High-quality, high-resolution medical imaging is essential for clinical care. Raman-based biomedical optical imaging uses non-ionizing infrared radiation to evaluate human tissues in real time and is used for early cancer detection, brain tumor diagnosis, and intraoperative tissue analysis. Unfortunately, optical imaging is vulnerable to image degradation due to laser scattering and absorption, which can result in diagnostic errors and misguided treatment. Restoration of optical images is a challenging computer vision task because the sources of image degradation are multi-factorial, stochastic, and tissue-dependent, preventing a straightforward method to obtain paired low-quality/high-quality data. Here, we present Restorative Step-Calibrated Diffusion (RSCD), an unpaired diffusion-based image restoration method that uses a step calibrator model to dynamically determine the number of steps required to complete the reverse diffusion process for image restoration. RSCD outperforms other widely used unpaired image restoration methods on both image quality and perceptual evaluation metrics for restoring optical images. Medical imaging experts consistently prefer images restored using RSCD in blinded comparison experiments and report minimal to no hallucinations. Finally, we show that RSCD improves performance on downstream clinical imaging tasks, including automated brain tumor diagnosis and deep tissue imaging. Our code is available at https://github.com/MLNeurosurg/restorative_step-calibrated_diffusion.

Figures (17)

• Figure 1: Examples of biomedical optical images restored using our proposed method, RSCD. Across a range of known and unknown sources of image degradation, RSCD provides high-quality image restoration of fresh, surgical specimens imaged during brain tumor surgery. Our method can restore optical images with severe image degradation such that, after restoration, they can be used for downstream clinical tasks, including automated brain tumor diagnosis and deep tissue imaging during surgery.
• Figure 2: An overview of Restorative Step-Calibrated Diffusion (RSCD). We view the low-quality image as the output of an incomplete diffusion generation process that starts from Gaussian noise ($t=T$) and performs $T$ steps of denoising (reverse diffusion) to generate a restored image at $t=0$. We use a step calibrator model to predict $t_{pred}$, the number of steps of diffusion model denoising needed for image restoration, and we perform the reverse diffusion starting from $t_{pred}$. In addition, we use dynamic recalibration to dynamically adjust the number of steps required for optimal image restoration, $t'_{pred}$. The dynamic recalibration process and subsequent $d$ steps of reverse diffusion denoising can be repeated until the restoration process runs to completion, obtaining the restored image at $t=0$.
• Figure 3: Importance of the Step calibrator. If we perform less than the optimal number of diffusion steps ($t < t_{pred}$), the image remains degraded. If we perform more than the needed steps ($t > t_{pred}$), the output image is excessively smooth, fine details are removed, and contains hallucinations (yellow arrow). ($t_{pred}=34$ for this image.)
• Figure 4: Visual comparison of unpaired image restoration methods. CycleGAN hallucinates/inpaints nuclei within non-cellular structures (yellow arrow). Synthetic noise and Deep Image Prior (DIP) produce overly smoothed, unrealistic images. Conditional diffusion and Regularized Reverse Diffusion (RRD) generally perform insufficient image restoration.
• Figure 5: Examples of paired low-quality/near-registered SRH images, and the restored image via RSCD.