Deep Learning CT Image Restoration using System Blur and Noise Models

TL;DR

This work tackles CT image restoration under varying blur and noise by introducing auxiliary inputs that encode system blur and noise properties. It integrates these priors into CNN blocks via input-variant, weight-variant, and mixed approaches within a U-Net framework, and evaluates across three degradation scenarios using PSNR. Results show that incorporating auxiliary information consistently improves restoration, with the mixed model delivering the strongest gains, particularly under combined blur and noise and anisotropic degradation. The approach provides a flexible, generalizable framework for conditioning deep networks on physical degradation models, with potential impact on diagnostic CT image quality and quantitative biomarker reliability.

Abstract

The restoration of images affected by blur and noise has been widely studied and has broad potential for applications including in medical imaging modalities like computed tomography (CT). Although the blur and noise in CT images can be attributed to a variety of system factors, these image properties can often be modeled and predicted accurately and used in classical restoration approaches for deconvolution and denoising. In classical approaches, simultaneous deconvolution and denoising can be challenging and often represent competing goals. Recently, deep learning approaches have demonstrated the potential to enhance image quality beyond classic limits; however, most deep learning models attempt a blind restoration problem and base their restoration on image inputs alone without direct knowledge of the image noise and blur properties. In this work, we present a method that leverages both degraded image inputs and a characterization of the system blur and noise to combine modeling and deep learning approaches. Different methods to integrate these auxiliary inputs are presented. Namely, an input-variant and a weight-variant approach wherein the auxiliary inputs are incorporated as a parameter vector before and after the convolutional block, respectively, allowing easy integration into any CNN architecture. The proposed model shows superior performance compared to baseline models lacking auxiliary inputs. Evaluations are based on the average Peak Signal-to-Noise Ratio (PSNR), selected examples of good and poor performance for varying approaches, and an input space analysis to assess the effect of different noise and blur on performance. Results demonstrate the efficacy of providing a deep learning model with auxiliary inputs, representing system blur and noise characteristics, to enhance the performance of the model in image restoration tasks.

Figures (7)

• Figure 1: Modified Model Structure. Our suggested changes focus on the convolutional block denoted by the black dashed rectangle, introducing three variations and two baselines (indicated by the blue arrow) to replace the conventional convolutional block (indicated by the orange arrows), with the exception of the final output layer where the standard block is retained. The input-variant and weight-variant models incorporate the auxiliary input prior to and following the convolutional process, respectively, while the mixed model utilizes it in both positions.
• Figure 2: PSNR Difference Histogram between Mixed Model and Baseline 2 across all scenarios. In each chart, regions indicating the top 10% (A), middle 10% (B), and bottom 10% (C) of PSNR improvement are highlighted in red, from which we select the displayed examples.
• Figure 3: Representative restoration examples for the variable blur scenario showing examples of the varied performance across methods. Note systematic oversharpening by baseline approaches in (A); as well as erroneous joining of noncontiguous features in (B).
• Figure 4: Representative restoration examples for the variable noise scenario. In (A) a regional mean attenuation is reported within the labeled square region of interest. Note that the bias in this region is variable across methods with the mixed approach having the lowest error. Small features present in the ground truth appear to be resolved when information about correlated noise is included in the model (B). In very challenging cases (C), it is possible for all models to create plausible but erroneous features.
• Figure 5: Representative restoration examples for the variable blur and noise scenario. The improvements associated with providing auxiliary blur and noise information to the network are particularly evident for anistropic blur functions on images with directional features. While significant improvements are possible, there is still the potential to miss such features in very challenging restoration examples.