Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CNN and ViT Efficiency Study on Tiny ImageNet and DermaMNIST Datasets

Aidar Amangeldi, Angsar Taigonyrov, Muhammad Huzaid Jawad, Chinedu Emmanuel Mbonu

TL;DR

This work compares CNN (ResNet-18) and Vision Transformer variants (ViT-Tiny/Small/Base/Large) on DermaMNIST and Tiny ImageNet to identify architectures that balance accuracy with low latency and memory footprint for edge deployment. By resizing inputs to 224×224, applying standardized augmentations, and tuning ViT configurations, the study finds ViT-Small with Patch16 often provides a near-Base level of accuracy while delivering 3–4× faster inference and about a 75% reduction in parameters relative to ViT-Base16, making it well-suited for mobile and drone applications. The results underscore a practical sweet spot: achieving high performance on small, low-resolution medical data with significantly lower compute, while retaining strong performance on general object datasets. The work also outlines current limitations (lack of on-device testing, limited model diversity, no compression or transfer learning) and proposes future directions such as edge benchmarking, model compression, and architectural refinements to further close the gap between accuracy and deployability.

Abstract

This study evaluates the trade-offs between convolutional and transformer-based architectures on both medical and general-purpose image classification benchmarks. We use ResNet-18 as our baseline and introduce a fine-tuning strategy applied to four Vision Transformer variants (Tiny, Small, Base, Large) on DermatologyMNIST and TinyImageNet. Our goal is to reduce inference latency and model complexity with acceptable accuracy degradation. Through systematic hyperparameter variations, we demonstrate that appropriately fine-tuned Vision Transformers can match or exceed the baseline's performance, achieve faster inference, and operate with fewer parameters, highlighting their viability for deployment in resource-constrained environments.

CNN and ViT Efficiency Study on Tiny ImageNet and DermaMNIST Datasets

TL;DR

This work compares CNN (ResNet-18) and Vision Transformer variants (ViT-Tiny/Small/Base/Large) on DermaMNIST and Tiny ImageNet to identify architectures that balance accuracy with low latency and memory footprint for edge deployment. By resizing inputs to 224×224, applying standardized augmentations, and tuning ViT configurations, the study finds ViT-Small with Patch16 often provides a near-Base level of accuracy while delivering 3–4× faster inference and about a 75% reduction in parameters relative to ViT-Base16, making it well-suited for mobile and drone applications. The results underscore a practical sweet spot: achieving high performance on small, low-resolution medical data with significantly lower compute, while retaining strong performance on general object datasets. The work also outlines current limitations (lack of on-device testing, limited model diversity, no compression or transfer learning) and proposes future directions such as edge benchmarking, model compression, and architectural refinements to further close the gap between accuracy and deployability.

Abstract

This study evaluates the trade-offs between convolutional and transformer-based architectures on both medical and general-purpose image classification benchmarks. We use ResNet-18 as our baseline and introduce a fine-tuning strategy applied to four Vision Transformer variants (Tiny, Small, Base, Large) on DermatologyMNIST and TinyImageNet. Our goal is to reduce inference latency and model complexity with acceptable accuracy degradation. Through systematic hyperparameter variations, we demonstrate that appropriately fine-tuned Vision Transformers can match or exceed the baseline's performance, achieve faster inference, and operate with fewer parameters, highlighting their viability for deployment in resource-constrained environments.
Paper Structure (32 sections, 5 figures, 6 tables)

This paper contains 32 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Dataset Overview
  3. Tiny ImageNet
  4. DermaMNIST
  5. Trends and Developments
  6. Skin Disease Classification
  7. General Object Classification
  8. Problems Identified
  9. Proposed Solution and Objectives
  10. Motivation and Related Work
  11. Background
  12. Vision Transformer
  13. ResNet
  14. Motivation
  15. Related Work
  16. ...and 17 more sections

Figures (5)

  • Figure 1: ViT Architecture (Deployed from b23)
  • Figure 2: The Network Architecture of ResNet18. Adapted from b12.
  • Figure 3: Accuracy comparison of ResNet18 and ViT on DermaMNIST vs. Tiny ImageNet, illustrating ViT's limitations on medical data and CNN's limitations on general datasets.
  • Figure 4: Overview of the Proposed Methodology.
  • Figure 5: Inference time vs. Accuracy, with marker size/color indicating the number of parameter count for two datasets. The red lines show the minimum amount for accuracy and inference time.