Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep learning EPI-TIRF cross-modality enables background subtraction and axial super-resolution for widefield fluorescence microscopy

Qiushi Li, Celi Lou, Yanfang Cheng, Bilang Gong, Xinlin Chen, Hao Chen, Baowan Li, Jieli Wang, Yulin Wang, Sipeng Yang, Yunqing Tang, Luru Dai

TL;DR

The paper tackles the fundamental limitation of wide-field fluorescence microscopy caused by out-of-focus background and low axial resolution. It presents ET2dNet, a physics-informed, cross-modality deep network that infers TIRF-like background subtraction and axial super-resolution from a single wide-field image using paired EPI-TIRF data and PSF-based self-supervision, with strong generalization across objectives. It further extends to ET3dNet, a 3D reconstruction network trained via knowledge distillation to produce artifact-reduced volumetric images from limited z-stacks, enabling fast, hardware-free axial imaging. Together, these methods provide a practical, adaptable tool for live-cell and histopathology applications, reducing hardware costs while delivering near-TIRF quality in both 2D and 3D reconstructions.

Abstract

The resolving ability of wide-field fluorescence microscopy is fundamentally limited by out-of-focus background owing to its low axial resolution, particularly for densely labeled biological samples. To address this, we developed ET2dNet, a deep learning-based EPI-TIRF cross-modality network that achieves TIRF-comparable background subtraction and axial super-resolution from a single wide-field image without requiring hardware modifications. The model employs a physics-informed hybrid architecture, synergizing supervised learning with registered EPI-TIRF image pairs and self-supervised physical modeling via convolution with the point spread function. This framework ensures exceptional generalization across microscope objectives, enabling few-shot adaptation to new imaging setups. Rigorous validation on cellular and tissue samples confirms ET2dNet's superiority in background suppression and axial resolution enhancement, while maintaining compatibility with deconvolution techniques for lateral resolution improvement. Furthermore, by extending this paradigm through knowledge distillation, we developed ET3dNet, a dedicated three-dimensional reconstruction network that produces artifact-reduced volumetric results. ET3dNet effectively removes out-of-focus background signals even when the input image stack lacks the source of background. This framework makes axial super-resolution imaging more accessible by providing an easy-to-deploy algorithm that avoids additional hardware costs and complexity, showing great potential for live cell studies and clinical histopathology.

Deep learning EPI-TIRF cross-modality enables background subtraction and axial super-resolution for widefield fluorescence microscopy

TL;DR

The paper tackles the fundamental limitation of wide-field fluorescence microscopy caused by out-of-focus background and low axial resolution. It presents ET2dNet, a physics-informed, cross-modality deep network that infers TIRF-like background subtraction and axial super-resolution from a single wide-field image using paired EPI-TIRF data and PSF-based self-supervision, with strong generalization across objectives. It further extends to ET3dNet, a 3D reconstruction network trained via knowledge distillation to produce artifact-reduced volumetric images from limited z-stacks, enabling fast, hardware-free axial imaging. Together, these methods provide a practical, adaptable tool for live-cell and histopathology applications, reducing hardware costs while delivering near-TIRF quality in both 2D and 3D reconstructions.

Abstract

The resolving ability of wide-field fluorescence microscopy is fundamentally limited by out-of-focus background owing to its low axial resolution, particularly for densely labeled biological samples. To address this, we developed ET2dNet, a deep learning-based EPI-TIRF cross-modality network that achieves TIRF-comparable background subtraction and axial super-resolution from a single wide-field image without requiring hardware modifications. The model employs a physics-informed hybrid architecture, synergizing supervised learning with registered EPI-TIRF image pairs and self-supervised physical modeling via convolution with the point spread function. This framework ensures exceptional generalization across microscope objectives, enabling few-shot adaptation to new imaging setups. Rigorous validation on cellular and tissue samples confirms ET2dNet's superiority in background suppression and axial resolution enhancement, while maintaining compatibility with deconvolution techniques for lateral resolution improvement. Furthermore, by extending this paradigm through knowledge distillation, we developed ET3dNet, a dedicated three-dimensional reconstruction network that produces artifact-reduced volumetric results. ET3dNet effectively removes out-of-focus background signals even when the input image stack lacks the source of background. This framework makes axial super-resolution imaging more accessible by providing an easy-to-deploy algorithm that avoids additional hardware costs and complexity, showing great potential for live cell studies and clinical histopathology.

Paper Structure

This paper contains 19 sections, 10 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Results
  3. EPI-TIRF cross-modality enables optical sectioning and axial super-resolution for a single WF image
  4. training 2D cross-modality network using paired EPI-TIRF images
  5. ET2dNet predictions show efficient background subtraction and axial-resolution enhancement
  6. Generalization of ET2dNet and few-shot learning
  7. Artifact-reduced 3D reconstruction network
  8. training a 3D reconstruction network with ET2dNet and EPI z-stack
  9. comparison of ET3dNet and SOTA 3D reconstruction algorithm
  10. Discussion
  11. Methods
  12. Microscope system
  13. Sample preparation
  14. Image Pre-processing
  15. Registration
  16. ...and 4 more sections

Figures (3)

  • Figure 1: EPI-TIRF cross-modality network for a single WF image.a EPI and TIRF illumination modes. b The architecture of ET2dNet. c Representative F-actin labeled C6 cell image of EPI(first column), Sparse-Deconvolution reconstruction (second column), Dark Sectioning reconstruction (third column), ET2dNet fake TIRF (fourth column), Sparse-Deconvolution reconstruction of ET2dNet fake TIRF (fifth column), and TIRF (sixth column). The magnified view of region in white box is displayed at the bottom. d EPI z-scan stack of white dashed box highlighted region in (c), with 200nm step size, and corresponding reconstruction. e Average MS-SSIM (to TIRF) of processed images in test dataset, by either Sparse-Deconvolutionm, Dark Sectioning and ET2dNet. f Representative images of human brain tissue section stained with an antibody against GFAP (provided by ref.wernersson_deconwolf_2024), EPI(first column), Sparse-Deconvolution reconstruction (second column), Dark Sectioning reconstruction (third column), ET2dNet fake TIRF (fourth column), Sparse-Deconvolution reconstruction of ET2dNet fake TIRF (fifth column), and confocal (sixth column). The magnified view of region in white box is displayed at the bottom, with marker line and corresponding intensity profile. Scale bar: 5 $\rm \mu m$.
  • Figure 2: Generalization and finetuning of ET2dNet.a Representative F-actin labeled C6 cell image of EPI(first column), ET2dNet 60X deconvolution reconstruction (second column), finetuned ET2dNet 100X deconvolution reconstruction (third column), and TIRF (fourth column). The magnified view of region in white box is displayed at the bottom. b Statistical comparisons of ET2dNet 60X/100X in terms of SSIM and PSNR across 20 EPI-TIRF image pairs. c Outputs of base and hybrid models at different fine-tuning epochs (0, 5, 10, 15), for the region in white box in (a). d Statistical comparisons of base (black) and hybrid (red) models in terms of MS-SSIM during the finetune(mean ± $\sigma$ confidence interval, smoothed over 3 epochs; 20 image pairs per model scale). e Representative images of human brain tissue section stained with an antibody against GFAP of EPI(first column), ET2dNet 60X output(second column), finetuned ET2dNet 100X output(third column), confocal(forth column), focus at 0 $\mu m$ (top) and 0.75 $\mu m$ (bottom). f The normalized intensity profile along the marker lines corresponding to (d). Scale bar: 6 $\rm \mu m$ in (e) and top of (a), 3 $\rm \mu m$ in (c) and bottom of (a).
  • Figure 3: Characterizations and demonstrations of ET3dNet.a The hybrid architecture of ET3dNet and the knowledge distillation process. b Representative xy (top) and xz (bottom) slice of F-actin labeled C6 cell stack, acquired by EPI(first column), DW 3D reconstruction(second column), Self-Net (third column), UniFMIR(fourth column), ET3dNet (fifth column), and Sparse Deconvolution reconstruction of ET3dNet output (sixth column). c Three-color lateral maximum-intensity projections of a fixed C6 cell, maximum-intensity projections (top) and representative xz slice (bottom), acquired by EPI(first column), DW 3D reconstruction(second column), Self-Net (third column), UniFMIR(fourth column), ET3dNet (fifth column), and Sparse Deconvolution reconstruction of ET3dNet output (sixth column). Red, F-actin labeled with Alexa Fluor™ 635 Phalloidin; blue, membrane labeled with Alexa Fluor™ 488 WGA; purple, nucleus stained by Hoechst 33258. d Representative xy slice of human brain tissue stack containing 6-layers, acquired by EPI(first column), DW (second column), confocal(fourth column), ET3dNet (fifth column), and Sparse Deconvolution reconstruction of ET3dNet output (sixth column). And DW of stacks containing 41-layers (third column). Scale bar: 10 $\rm \mu m$.