Cryo-CARE: Content-Aware Image Restoration for Cryo-Transmission Electron Microscopy Data

TL;DR

This work dramatically increases contrast in cryo-TEM images, which improves the interpretability of the acquired data and shows that automated downstream processing on restored image data, demonstrated on a dense segmentation task, leads to improved results.

Abstract

Multiple approaches to use deep learning for image restoration have recently been proposed. Training such approaches requires well registered pairs of high and low quality images. While this is easily achievable for many imaging modalities, e.g. fluorescence light microscopy, for others it is not. Cryo-transmission electron microscopy (cryo-TEM) could profoundly benefit from improved denoising methods, unfortunately it is one of the latter. Here we show how recent advances in network training for image restoration tasks, i.e. denoising, can be applied to cryo-TEM data. We describe our proposed method and show how it can be applied to single cryo-TEM projections and whole cryo-tomographic image volumes. Our proposed restoration method dramatically increases contrast in cryo-TEM images, which improves the interpretability of the acquired data. Furthermore we show that automated downstream processing on restored image data, demonstrated on a dense segmentation task, leads to improved results.

Figures (5)

• Figure 1: Cryo-CARE results on a 2D cryo-TEM projection. Subfigures and insets show: raw input data (a), median filtered restoration baseline (b), Cryo-CARE results when trained on tomographic tilt-angle pairs (c), on acquired image pairs (d), and on dose-fractionated movie frames (e).
• Figure 2: Cryo-CARE results on a 3D cryo-TEM tomogram. Subfigures show: a section through the raw tomogram (a), the non-linear anisotropic diffusion filtered baseline (b), cryo-CARE results when trained via our proposed T2T-eoa (c) and T2T-df (d). The graph shows the Fourier shell correlation (FSC) curves of the raw tomogram, the baseline, and our proposed methods.
• Figure 3: Cryo-CARE restoration on the publicly available EMPIAR-10110 dataset. (a) Raw projection (single tilt angle). (b) Median filtered baseline. (c) Our P2P-tap results. (d) Raw tomogram. (e) NAD filtered baseline. (d) Our even-odd T2T restoration.
• Figure 4: Tomogram reconstruction artifacts. Tomograms reconstructed from P2P restored tilt-angles lead to strong missing-wedge artifacts (a). This problem is reduced using our proposed T2T training scheme (b).
• Figure 5: Automated downstream analysis on raw data (a) and a T2T-df restored tomogram (b). Ground-truth voxels are shown in violet, true-positives in turquoise, and false-positives in orange. Precision-recall plots on increasing segment size threshold (see main text) are shown below. The pentagons correspond to sub-figures (a) and (b).