Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Building Efficient Lightweight CNN Models

Nathan Isong

TL;DR

This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy, and integrates two stages of training; dual-input-output model and transfer learning with progressive unfreezing.

Abstract

Convolutional Neural Networks (CNNs) are pivotal in image classification tasks due to their robust feature extraction capabilities. However, their high computational and memory requirements pose challenges for deployment in resource-constrained environments. This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy. The approach integrates two stages of training; dual-input-output model and transfer learning with progressive unfreezing. The dual-input-output model train on original and augmented datasets, enhancing robustness. Progressive unfreezing is applied to the unified model to optimize pre-learned features during fine-tuning, enabling faster convergence and improved model accuracy. The methodology was evaluated on three benchmark datasets; handwritten digit MNIST, fashion MNIST, and CIFAR-10. The proposed model achieved a state-of-the-art accuracy of 99% on the handwritten digit MNIST and 89% on fashion MNIST, with only 14,862 parameters and a model size of 0.17 MB. While performance on CIFAR-10 was comparatively lower (65% with less than 20,00 parameters), the results highlight the scalability of this method. The final model demonstrated fast inference times and low latency, making it suitable for real-time applications. Future directions include exploring advanced augmentation techniques, improving architectural scalability for complex datasets, and extending the methodology to tasks beyond classification. This research underscores the potential for creating efficient, scalable, and task-specific CNNs for diverse applications.

Building Efficient Lightweight CNN Models

TL;DR

This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy, and integrates two stages of training; dual-input-output model and transfer learning with progressive unfreezing.

Abstract

Convolutional Neural Networks (CNNs) are pivotal in image classification tasks due to their robust feature extraction capabilities. However, their high computational and memory requirements pose challenges for deployment in resource-constrained environments. This paper introduces a methodology to construct lightweight CNNs while maintaining competitive accuracy. The approach integrates two stages of training; dual-input-output model and transfer learning with progressive unfreezing. The dual-input-output model train on original and augmented datasets, enhancing robustness. Progressive unfreezing is applied to the unified model to optimize pre-learned features during fine-tuning, enabling faster convergence and improved model accuracy. The methodology was evaluated on three benchmark datasets; handwritten digit MNIST, fashion MNIST, and CIFAR-10. The proposed model achieved a state-of-the-art accuracy of 99% on the handwritten digit MNIST and 89% on fashion MNIST, with only 14,862 parameters and a model size of 0.17 MB. While performance on CIFAR-10 was comparatively lower (65% with less than 20,00 parameters), the results highlight the scalability of this method. The final model demonstrated fast inference times and low latency, making it suitable for real-time applications. Future directions include exploring advanced augmentation techniques, improving architectural scalability for complex datasets, and extending the methodology to tasks beyond classification. This research underscores the potential for creating efficient, scalable, and task-specific CNNs for diverse applications.
Paper Structure (19 sections, 16 figures, 6 tables)

This paper contains 19 sections, 16 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Datasets
  5. Data Preparation and Transformation
  6. Model Architecture
  7. Model Compilation and Callbacks
  8. Model Training
  9. First Training Stage
  10. Creating the Final Model
  11. Second (Final) Training Stage
  12. Results and Discussion
  13. Training Performance
  14. Model Lightweightness
  15. Model Latency
  16. ...and 4 more sections

Figures (16)

  • Figure 1: Flow Chart showing the methodology step.
  • Figure 2: Structure of model2 (identical to model1) before concatenation.
  • Figure 3: Concatenated model structure: here we have the inputs (input_layer_5, input_layer_4), the models (model1 f, model2 f) as described in Figure \ref{['fig:model1-model2']}, and the outputs (model1, model2).
  • Figure 4: First training performance metrics of the two submodels for the handwritten MNIST dataset.
  • Figure 5: First training performance metrics of the two submodels for the fashion MNIST.
  • ...and 11 more figures