Digital staining in optical microscopy using deep learning -- a review

TL;DR

An in-depth analysis of the current state-of-the-art in digital staining, methods of good practice, identify pitfalls and challenges and postulate promising advances towards potential future implementations and applications are provided.

Abstract

Until recently, conventional biochemical staining had the undisputed status as well-established benchmark for most biomedical problems related to clinical diagnostics, fundamental research and biotechnology. Despite this role as gold-standard, staining protocols face several challenges, such as a need for extensive, manual processing of samples, substantial time delays, altered tissue homeostasis, limited choice of contrast agents for a given sample, 2D imaging instead of 3D tomography and many more. Label-free optical technologies, on the other hand, do not rely on exogenous and artificial markers, by exploiting intrinsic optical contrast mechanisms, where the specificity is typically less obvious to the human observer. Over the past few years, digital staining has emerged as a promising concept to use modern deep learning for the translation from optical contrast to established biochemical contrast of actual stainings. In this review article, we provide an in-depth analysis of the current state-of-the-art in this field, suggest methods of good practice, identify pitfalls and challenges and postulate promising advances towards potential future implementations and applications.

Figures (6)

• Figure 1: Basic principle of Digital staining (a) Example data showing phase contrast microscopy images as input to predict fluorescence images from specific antibody staining as ground truth. Images show human motor neurons stained for nuclei (DAPI), dendrites (anti-MAP2) and axons (anti-neurofilament). Ground truth image were acquired on confocal fluorescence microscopy (scale bar: 100 µ m). Data available at https://github.com/google/in-silico-labeling from Ref. 60 with permission from Elsevier and Copyright Clearance Center. (b) The general supervised machine learning workflow for digital staining. (c) The most commonly used models: besides earlier implementations of color-coding with a linear contrast translation equation f(k) or feature engineering and classical ML, almost all modern digital staining implementations use deep learning with either CNN and GAN architectures.
• Figure 2: Pairings of input and target contrast Target image contrast is plotted against the input contrast, with the number of publications for each combination as a color-coded heat map. IHC = immuno-histochemcial stain, IF = immuno-fluorescence stain, WF = wide field (white light illumination), AF = autofluorescence, PAM = photo-acoustic microscopy, IR = infra-red. An extended version including a detailed literature analysis can be found in the supplementary material of this manuscript, see suppl.fig. \ref{['fig:sup_fig2']}
• Figure 3: Generation of image pairs for the training of digital staining models (A-E) Schematic workflow of the five different procedures. The table shows positive features (green), neutral features (orange) and negative features (red).
• Figure 4: Historical trends in the field of digital staining. (a) The total number of publications in the field (b-h) and grouped according to the categories.
• Figure S1: Parallel categories with connections All reviewed articles as parallel and linked categories. The year of each publication is color-coded. An interactive version of this plot is available as supplementary html file.