Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatically Score Tissue Images Like a Pathologist by Transfer Learning

Iris Yan

TL;DR

The proposed algorithm has made it possible to break the critical accuracy barrier (the 75% accuracy level of pathologists), with a reported accuracy of 75.9% on breast cancer TMA images from the Stanford Tissue Microarray Database.

Abstract

Cancer is the second leading cause of death in the world. Diagnosing cancer early on can save many lives. Pathologists have to look at tissue microarray (TMA) images manually to identify tumors, which can be time-consuming, inconsistent and subjective. Existing automatic algorithms either have not achieved the accuracy level of a pathologist or require substantial human involvements. A major challenge is that TMA images with different shapes, sizes, and locations can have the same score. Learning staining patterns in TMA images requires a huge number of images, which are severely limited due to privacy and regulation concerns in medical organizations. TMA images from different cancer types may share certain common characteristics, but combining them directly harms the accuracy due to heterogeneity in their staining patterns. Transfer learning is an emerging learning paradigm that allows borrowing strength from similar problems. However, existing approaches typically require a large sample from similar learning problems, while TMA images of different cancer types are often available in small sample size and further existing algorithms are limited to transfer learning from one similar problem. We propose a new transfer learning algorithm that could learn from multiple related problems, where each problem has a small sample and can have a substantially different distribution from the original one. The proposed algorithm has made it possible to break the critical accuracy barrier (the 75% accuracy level of pathologists), with a reported accuracy of 75.9% on breast cancer TMA images from the Stanford Tissue Microarray Database. It is supported by recent developments in transfer learning theory and empirical evidence in clustering technology. This will allow pathologists to confidently adopt automatic algorithms in recognizing tumors consistently with a higher accuracy in real time.

Automatically Score Tissue Images Like a Pathologist by Transfer Learning

TL;DR

The proposed algorithm has made it possible to break the critical accuracy barrier (the 75% accuracy level of pathologists), with a reported accuracy of 75.9% on breast cancer TMA images from the Stanford Tissue Microarray Database.

Abstract

Cancer is the second leading cause of death in the world. Diagnosing cancer early on can save many lives. Pathologists have to look at tissue microarray (TMA) images manually to identify tumors, which can be time-consuming, inconsistent and subjective. Existing automatic algorithms either have not achieved the accuracy level of a pathologist or require substantial human involvements. A major challenge is that TMA images with different shapes, sizes, and locations can have the same score. Learning staining patterns in TMA images requires a huge number of images, which are severely limited due to privacy and regulation concerns in medical organizations. TMA images from different cancer types may share certain common characteristics, but combining them directly harms the accuracy due to heterogeneity in their staining patterns. Transfer learning is an emerging learning paradigm that allows borrowing strength from similar problems. However, existing approaches typically require a large sample from similar learning problems, while TMA images of different cancer types are often available in small sample size and further existing algorithms are limited to transfer learning from one similar problem. We propose a new transfer learning algorithm that could learn from multiple related problems, where each problem has a small sample and can have a substantially different distribution from the original one. The proposed algorithm has made it possible to break the critical accuracy barrier (the 75% accuracy level of pathologists), with a reported accuracy of 75.9% on breast cancer TMA images from the Stanford Tissue Microarray Database. It is supported by recent developments in transfer learning theory and empirical evidence in clustering technology. This will allow pathologists to confidently adopt automatic algorithms in recognizing tumors consistently with a higher accuracy in real time.
Paper Structure (11 sections, 3 equations, 7 figures, 2 algorithms)

This paper contains 11 sections, 3 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Methods
  3. The spatial histogram
  4. Random Forests
  5. Transfer learning
  6. An algorithmic description
  7. Experiments and results
  8. Results
  9. Understanding the transfer learning scheme
  10. Conclusions
  11. Acknowledgments

Figures (7)

  • Figure 1: The staining patterns vary highly across TMA images. Images with the same score can look drastically different.
  • Figure 2: Illustration of the overall flow of the proposed algorithm. The '+' sign stands for combining images from multiple sources. ER is the name of biomarker associated with the target cancer type and we use it to indicate the corresponding TMA images, while NMB, CK56 and CD117 are those for other cancer types.
  • Figure 3: Illustration of a toy image and the spatial histogram matrix. The left panel stands for the original image where the numbers are the gray values, and the right is the resulting spatial histogram matrix with a dimension $4 \times 4$. The two diagonally (i.e., along 45 ° direction) neighboring pixels with both gray levels of 1 occur twice in the image, so the (1, 1)-entry of the spatial histogram matrix has a value of 2.
  • Figure 4: Illustration of transfer learning. ER is the name of biomarker associated with the target cancer type, while NMB, CK56 and CD117 are those for other cancer types.
  • Figure 5: Example transferable images from other cancer types. The left 3 columns of images are TMA images for breast cancer and indicated by the associated biomarker estrigen receptor (ER). The right 3 columns are TMA images for cancer types indicated by biomarkers NMB, CK56 and CD117, respectively, which have a similar appearance as those for ER with the same label.
  • ...and 2 more figures