Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automating Parameter Selection in Deep Image Prior for Fluorescence Microscopy Image Denoising via Similarity-Based Parameter Transfer

Lina Meyer, Felix Wissel, Tobias Knopp, Susanne Pfefferle, Ralf Fliegert, Maximilian Sandmann, Liana Uebler, Franziska Möckl, Björn-Philipp Diercks, David Lohr, René Werner

TL;DR

The paper tackles the challenge of selecting DIP parameters for fluorescence microscopy denoising, which traditionally requires extensive per-image tuning. It introduces AUTO-DIP, a similarity-based parameter transfer framework that jointly optimizes DIP architecture and stopping point by leveraging a calibrated image set and transferring parameters to new images. Across multiple public datasets, AUTO-DIP with group-based microscope-specimen similarity delivers near-optimal denoising while reducing runtime by roughly a factor of three compared with the original DIP, and it remains competitive with state-of-the-art unsupervised variational methods. This approach broadens the practical applicability of DIP in microscopy where ground-truth data are scarce and supervised models are impractical.

Abstract

Unsupervised deep image prior (DIP) addresses shortcomings of training data requirements and limited generalization associated with supervised deep learning. The performance of DIP depends on the network architecture and the stopping point of its iterative process. Optimizing these parameters for a new image requires time, restricting DIP application in domains where many images need to be processed. Focusing on fluorescence microscopy data, we hypothesize that similar images share comparable optimal parameter configurations for DIP-based denoising, potentially enabling optimization-free DIP for fluorescence microscopy. We generated a calibration (n=110) and validation set (n=55) of semantically different images from an open-source dataset for a network architecture search targeted towards ideal U-net architectures and stopping points. The calibration set represented our transfer basis. The validation set enabled the assessment of which image similarity criterion yields the best results. We then implemented AUTO-DIP, a pipeline for automatic parameter transfer, and compared it to the originally published DIP configuration (baseline) and a state-of-the-art image-specific variational denoising approach. We show that a parameter transfer from the calibration dataset to a test image based on only image metadata similarity (e.g., microscope type, imaged specimen) leads to similar and better performance than a transfer based on quantitative image similarity measures. AUTO-DIP outperforms the baseline DIP (DIP with original DIP parameters) as well as the variational denoising approaches for several open-source test datasets of varying complexity, particularly for very noisy inputs. Applications to locally acquired fluorescence microscopy images further proved superiority of AUTO-DIP.

Automating Parameter Selection in Deep Image Prior for Fluorescence Microscopy Image Denoising via Similarity-Based Parameter Transfer

TL;DR

The paper tackles the challenge of selecting DIP parameters for fluorescence microscopy denoising, which traditionally requires extensive per-image tuning. It introduces AUTO-DIP, a similarity-based parameter transfer framework that jointly optimizes DIP architecture and stopping point by leveraging a calibrated image set and transferring parameters to new images. Across multiple public datasets, AUTO-DIP with group-based microscope-specimen similarity delivers near-optimal denoising while reducing runtime by roughly a factor of three compared with the original DIP, and it remains competitive with state-of-the-art unsupervised variational methods. This approach broadens the practical applicability of DIP in microscopy where ground-truth data are scarce and supervised models are impractical.

Abstract

Unsupervised deep image prior (DIP) addresses shortcomings of training data requirements and limited generalization associated with supervised deep learning. The performance of DIP depends on the network architecture and the stopping point of its iterative process. Optimizing these parameters for a new image requires time, restricting DIP application in domains where many images need to be processed. Focusing on fluorescence microscopy data, we hypothesize that similar images share comparable optimal parameter configurations for DIP-based denoising, potentially enabling optimization-free DIP for fluorescence microscopy. We generated a calibration (n=110) and validation set (n=55) of semantically different images from an open-source dataset for a network architecture search targeted towards ideal U-net architectures and stopping points. The calibration set represented our transfer basis. The validation set enabled the assessment of which image similarity criterion yields the best results. We then implemented AUTO-DIP, a pipeline for automatic parameter transfer, and compared it to the originally published DIP configuration (baseline) and a state-of-the-art image-specific variational denoising approach. We show that a parameter transfer from the calibration dataset to a test image based on only image metadata similarity (e.g., microscope type, imaged specimen) leads to similar and better performance than a transfer based on quantitative image similarity measures. AUTO-DIP outperforms the baseline DIP (DIP with original DIP parameters) as well as the variational denoising approaches for several open-source test datasets of varying complexity, particularly for very noisy inputs. Applications to locally acquired fluorescence microscopy images further proved superiority of AUTO-DIP.
Paper Structure (22 sections, 2 equations, 4 figures, 2 tables)

This paper contains 22 sections, 2 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Deep Image Prior and Related Work
  4. Our Approach: AUTO-DIP
  5. Calibration and Validation Dataset Definition
  6. Microscopy Datasets
  7. FMD
  8. Hagen
  9. BioSR
  10. W2S
  11. Shah
  12. UKE-internal Data
  13. Data Preprocessing
  14. Experiments
  15. Optimal Parameter and Evaluation Measure Selection
  16. ...and 7 more sections

Figures (4)

  • Figure 1: (a) Example results for the validation dataset. From left to right: noisy input image, result with original DIP configuration, AUTO-DIP result with transferred parameters using Group-Based Similarity, best result achievable from all parameters of the calibration set according to the combined LPIPS-PSNR measurement, ground truth image. Each row shows a roi from a different image, covering the three structures and three microscope types contained of the training data. (b) Relationship of image complexity, measured by the mean image gradient, and optimal number of layers (left) and number of channels (right) for DIP-based denoising for the calibration set. The number n denotes the number of images for which the corresponding depth and width are found optimal in terms of the combined measurement of PSNR and LPIPS. Abbreviations: Conf -- Confocal, Wf -- Widefield, TP -- Two-Photon
  • Figure 2: Generalization results for various test sets. From left to right: noisy input image, result from sparsity-based denoising, result with original DIP configuration, result with AUTO-DIP, ground truth image. First row: roi of a confocal microscopy image of a fixed BPAE cell from the FMD dataset with three channels depicting mitochondria (red), actin (green), and nuclei (blue). Rows 2-4: single-channel grey-scale images of the above image. Rows 5-6: confocal microscopy of zebrafish embryo and two-photon microscopy of mouse brain tissue from the FMD dataset. Rows 7-8: widefield microscopy of CCPs and membrane from the BioSR, and Hagen datasets. Abbreviations: BPAE -- bpae, CCPs -- ccp, Mito -- Mitochondria
  • Figure 3: Impact of noise level on denoising performance illustrated for BioSR and Hagen data. From left to right: noisy input image; output of sparsity-based variational denoising; original DIP configuration; AUTO-DIP output; ground truth image. First two rows show images of endoplasmatic reticulum (ER) with high noise level (row 1) and low noise level (row 2). Last two rows show actin with high noise level (row 3) and low noise level (row 4).
  • Figure 4: AUTO-DIP denoising results for an A549-A/T cell infected with SARS-CoV-2. From left to right: noisy input image; result with original DIP configuration; AUTO-DIP result with parameter transfer based on the microscope type (here: confocal microscope); AUTO-DIP result with parameter transfer based on UMAP-based semantic image similarity. First row: three channel image of the infected A549-A/T cell with marked roi. Channels depict nuclei (blue, 405 nm), the Spike protein (green, 488 nm), and Nsp3 (red, 561 nm). Second row: roi for the combined channels. Rows 3-5: single-channel images for the roi.