Unsupervised Denoising for Signal-Dependent and Row-Correlated Imaging Noise

TL;DR

This work tackles the challenge of denoising microscopy images affected by signal-dependent, row- or column-correlated noise without requiring paired clean data or a noise model. It proposes a hierarchical VAE whose autoregressive decoder is restricted to axis-aligned receptive fields, so the latent variables capture the clean signal while the decoder models the noise. A separate signal-decoder maps latent samples back to denoised images, trained using only noisy data in a self-supervised fashion. Across multiple microscopy modalities, the method achieves state-of-the-art unsupervised performance and often surpasses supervised baselines, enabling practical denoising when clean references are unavailable.

Abstract

Accurate analysis of microscopy images is hindered by the presence of noise. This noise is usually signal-dependent and often additionally correlated along rows or columns of pixels. Current self- and unsupervised denoisers can address signal-dependent noise, but none can reliably remove noise that is also row- or column-correlated. Here, we present the first fully unsupervised deep learning-based denoiser capable of handling imaging noise that is row-correlated as well as signal-dependent. Our approach uses a Variational Autoencoder (VAE) with a specially designed autoregressive decoder. This decoder is capable of modeling row-correlated and signal-dependent noise but is incapable of independently modeling underlying clean signal. The VAE therefore produces latent variables containing only clean signal information, and these are mapped back into image space using a proposed second decoder network. Our method does not require a pre-trained noise model and can be trained from scratch using unpaired noisy data. We benchmark our approach on microscopy datatsets from a range of imaging modalities and sensor types, each with row- or column-correlated, signal-dependent noise, and show that it outperforms existing self- and unsupervised denoisers.

Figures (4)

• Figure 2: a): A Variational Autoencoder chenvariational (solid arrows) is trained to model the distribution of noisy images $\mathbf{x}$. The autoregressive (AR) decoder models the noise component of the images, while the latent variable models only the clean signal component $\mathbf{s}$. In a second step (dashed arrows), our novel signal decoder is trained to map latent variables into image space, producing an estimate of the signal underlying $\mathbf{x}$. b): To ensure that the decoder models only the imaging noise and the latent variables capture only the signal, we modify the AR decoder's receptive field. In a full AR receptive field (\ref{['eq:full-auto']}), each output pixel (red) is a function of all input pixels located above and to the left (blue). In our decoder's row-based AR receptive field (\ref{['eq:row_decoder']}), each output pixel is a function of input pixels located in the same row, which corresponds to the row-correlated structure of imaging noise.
• Figure 3: Visual results from our method and two unpaired baselines on all datasets. The spatial autocorrelation of the noise is overlaid on each noisy image, with red indicating a positive correlation and blue indicating a negative correlation. The direction of the correlation is given by the orientation of the autocorrelation bar. Additionally, the signal dependence of the noise in each dataset is shown in the right-hand column. The horizontal axis of these graphs is the clean signal intensity as a percentage of the maximum, while the vertical axis is the variance of noisy pixel values recorded for these signal intensities. Ground truth must be used to calculate signal dependence, so orange lines are used where denoised images from our method are used as pseudo-ground truth.
• Figure 4: The FFHQ - Checkerboard dataset wass denoised 5 times by varying the number of pixels covered by the AR decoder's receptive field. The images show denoising results for each receptive field size with PSNR overlaid. Additionally, an image denoised using a full AR receptive field is included on the right. In this situation, the signal decoder is given completely uninformative inputs and learns to output the mean of the training dataset.
• Figure 5: A noisy image from the Convallaria A dataset was encoded and decoded to produce a reconstructed observation and an artificial noise sample. The sampled noise has spatial autocorrelation and signal dependence that match those of the real noise, indicating that the AR decoder has learnt an accurate noise model.