Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comparative Study of CNN, ResNet, and Vision Transformers for Multi-Classification of Chest Diseases

Ananya Jain, Aviral Bhardwaj, Kaushik Murali, Isha Surani

TL;DR

This paper compares CNN, ResNet, and Vision Transformer (ViT) architectures for multi-label chest disease classification on the NIH Chest X-ray dataset, evaluating variants including ViT-v1/32, ViT-v2/32, and ViT-ResNet/16 with and without ImageNet-21k pretraining. Using a subset of 85,000 images and data augmentation, the study reports that pre-trained ViTs outperform the CNN baseline, while ResNet remains highly competitive; the ViT-ResNet/16 hybrid achieves similar or better ROC characteristics with faster training. Attention-map visualizations provide interpretability by highlighting image regions influencing predictions. Limitations include dataset size and class imbalance, suggesting that larger, disease-focused pretraining could further improve clinical utility and integration into radiology workflows.

Abstract

Large language models, notably utilizing Transformer architectures, have emerged as powerful tools due to their scalability and ability to process large amounts of data. Dosovitskiy et al. expanded this architecture to introduce Vision Transformers (ViT), extending its applicability to image processing tasks. Motivated by this advancement, we fine-tuned two variants of ViT models, one pre-trained on ImageNet and another trained from scratch, using the NIH Chest X-ray dataset containing over 100,000 frontal-view X-ray images. Our study evaluates the performance of these models in the multi-label classification of 14 distinct diseases, while using Convolutional Neural Networks (CNNs) and ResNet architectures as baseline models for comparison. Through rigorous assessment based on accuracy metrics, we identify that the pre-trained ViT model surpasses CNNs and ResNet in this multilabel classification task, highlighting its potential for accurate diagnosis of various lung conditions from chest X-ray images.

A Comparative Study of CNN, ResNet, and Vision Transformers for Multi-Classification of Chest Diseases

TL;DR

This paper compares CNN, ResNet, and Vision Transformer (ViT) architectures for multi-label chest disease classification on the NIH Chest X-ray dataset, evaluating variants including ViT-v1/32, ViT-v2/32, and ViT-ResNet/16 with and without ImageNet-21k pretraining. Using a subset of 85,000 images and data augmentation, the study reports that pre-trained ViTs outperform the CNN baseline, while ResNet remains highly competitive; the ViT-ResNet/16 hybrid achieves similar or better ROC characteristics with faster training. Attention-map visualizations provide interpretability by highlighting image regions influencing predictions. Limitations include dataset size and class imbalance, suggesting that larger, disease-focused pretraining could further improve clinical utility and integration into radiology workflows.

Abstract

Large language models, notably utilizing Transformer architectures, have emerged as powerful tools due to their scalability and ability to process large amounts of data. Dosovitskiy et al. expanded this architecture to introduce Vision Transformers (ViT), extending its applicability to image processing tasks. Motivated by this advancement, we fine-tuned two variants of ViT models, one pre-trained on ImageNet and another trained from scratch, using the NIH Chest X-ray dataset containing over 100,000 frontal-view X-ray images. Our study evaluates the performance of these models in the multi-label classification of 14 distinct diseases, while using Convolutional Neural Networks (CNNs) and ResNet architectures as baseline models for comparison. Through rigorous assessment based on accuracy metrics, we identify that the pre-trained ViT model surpasses CNNs and ResNet in this multilabel classification task, highlighting its potential for accurate diagnosis of various lung conditions from chest X-ray images.
Paper Structure (17 sections, 6 figures, 2 tables)

This paper contains 17 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. CNN
  5. Residual Networks (ResNet) \ref{['ResNet']}
  6. Vision Transformers (ViTs)\ref{['ViT']}
  7. Experiment
  8. Dataset
  9. Models
  10. Evaluation Metrics
  11. Results
  12. Discussion
  13. AUC and ROC Curve
  14. Attention Maps
  15. Limitations
  16. ...and 2 more sections

Figures (6)

  • Figure 1: ViT, and ResNet Model Illustration
  • Figure 2: ViT-v1/32 and ViT-v2/32 Input
  • Figure 3: Training and Validation Accuracy for various Model on NIH Chest X-Ray Images
  • Figure 4: ViT-v2 Training and Validation Loss & Accuracy
  • Figure 5: Attention map of a No Finding image
  • ...and 1 more figures