Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GPC: Generative and General Pathology Image Classifier

Anh Tien Nguyen, Jin Tae Kwak

TL;DR

This work addresses the scalability challenge of task-specific pathology classifiers by introducing GPC, a single model that learns from diverse pathology datasets to perform multiple classification tasks through a generative, image-to-text framework. GPC combines a ConvNeXt-based CNN feature extractor, a three-layer MLP projector, and a decoder-only Transformer language model (OPT) to output textual class labels conditioned on image features, enabling cross-task transfer. Across six datasets and four tasks, GPC demonstrates competitive performance, often ranking at or near the top, with the task-agnostic generative setup (E_TAG) showing promise while standard task-agnostic classifiers (E_TA) lag behind. The results underscore the potential of generative, universal pathology classifiers for scalable, multi-task image analysis, with future work extending to more organs and tasks and improving efficiency of generative decoding.

Abstract

Deep learning has been increasingly incorporated into various computational pathology applications to improve its efficiency, accuracy, and robustness. Although successful, most previous approaches for image classification have crucial drawbacks. There exist numerous tasks in pathology, but one needs to build a model per task, i.e., a task-specific model, thereby increasing the number of models, training resources, and cost. Moreover, transferring arbitrary task-specific model to another task is still a challenging problem. Herein, we propose a task-agnostic generative and general pathology image classifier, so called GPC, that aims at learning from diverse kinds of pathology images and conducting numerous classification tasks in a unified model. GPC, equipped with a convolutional neural network and a Transformer-based language model, maps pathology images into a high-dimensional feature space and generates pertinent class labels as texts via the image-to-text classification mechanism. We evaluate GPC on six datasets for four different pathology image classification tasks. Experimental results show that GPC holds considerable potential for developing an effective and efficient universal model for pathology image analysis.

GPC: Generative and General Pathology Image Classifier

TL;DR

This work addresses the scalability challenge of task-specific pathology classifiers by introducing GPC, a single model that learns from diverse pathology datasets to perform multiple classification tasks through a generative, image-to-text framework. GPC combines a ConvNeXt-based CNN feature extractor, a three-layer MLP projector, and a decoder-only Transformer language model (OPT) to output textual class labels conditioned on image features, enabling cross-task transfer. Across six datasets and four tasks, GPC demonstrates competitive performance, often ranking at or near the top, with the task-agnostic generative setup (E_TAG) showing promise while standard task-agnostic classifiers (E_TA) lag behind. The results underscore the potential of generative, universal pathology classifiers for scalable, multi-task image analysis, with future work extending to more organs and tasks and improving efficiency of generative decoding.

Abstract

Deep learning has been increasingly incorporated into various computational pathology applications to improve its efficiency, accuracy, and robustness. Although successful, most previous approaches for image classification have crucial drawbacks. There exist numerous tasks in pathology, but one needs to build a model per task, i.e., a task-specific model, thereby increasing the number of models, training resources, and cost. Moreover, transferring arbitrary task-specific model to another task is still a challenging problem. Herein, we propose a task-agnostic generative and general pathology image classifier, so called GPC, that aims at learning from diverse kinds of pathology images and conducting numerous classification tasks in a unified model. GPC, equipped with a convolutional neural network and a Transformer-based language model, maps pathology images into a high-dimensional feature space and generates pertinent class labels as texts via the image-to-text classification mechanism. We evaluate GPC on six datasets for four different pathology image classification tasks. Experimental results show that GPC holds considerable potential for developing an effective and efficient universal model for pathology image analysis.
Paper Structure (20 sections, 3 equations, 4 figures, 4 tables)

This paper contains 20 sections, 3 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem formulation
  4. Network architecture
  5. Feature extractor
  6. Projector
  7. Language model
  8. Experiments
  9. Datasets
  10. Colorectal cancer grading
  11. Prostate cancer grading
  12. Gastric cancer grading
  13. Colorectal tissue typing
  14. Comparative models
  15. Experimental design
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Overview of GPC.
  • Figure 2: Three types of classification approaches. (a) Task-specific model, (b) Task-agnostic model, and (c) Task-agnostic generative model.
  • Figure 3: Examples of correct predictions by GPC. Pred denotes prediction.
  • Figure 4: Examples of incorrect predictions by GPC. Pred and GT denote a prediction and a ground truth, respectively.