Training a Custom CNN on Five Heterogeneous Image Datasets
Anika Tabassum, Tasnuva Mahazabin Tuba, Nafisa Naznin
TL;DR
This work analyzes a lightweight custom CNN against scratch-trained ResNet-18/VGG-16 and ImageNet-based transfer learning across five diverse datasets from agricultural and urban contexts. It demonstrates that transfer learning typically yields better generalization on limited data, while the custom CNN offers a compact and efficient baseline. Scratch-trained deep models often underperform when data are scarce or imbalanced, underscoring the value of pre-trained representations for real-world visual classification tasks. The findings provide practical guidance for deploying deep learning in resource-constrained environments, with implications for smart cities and agricultural automation.
Abstract
Deep learning has transformed visual data analysis, with Convolutional Neural Networks (CNNs) becoming highly effective in learning meaningful feature representations directly from images. Unlike traditional manual feature engineering methods, CNNs automatically extract hierarchical visual patterns, enabling strong performance across diverse real-world contexts. This study investigates the effectiveness of CNN-based architectures across five heterogeneous datasets spanning agricultural and urban domains: mango variety classification, paddy variety identification, road surface condition assessment, auto-rickshaw detection, and footpath encroachment monitoring. These datasets introduce varying challenges, including differences in illumination, resolution, environmental complexity, and class imbalance, necessitating adaptable and robust learning models. We evaluate a lightweight, task-specific custom CNN alongside established deep architectures, including ResNet-18 and VGG-16, trained both from scratch and using transfer learning. Through systematic preprocessing, augmentation, and controlled experimentation, we analyze how architectural complexity, model depth, and pre-training influence convergence, generalization, and performance across datasets of differing scale and difficulty. The key contributions of this work are: (1) the development of an efficient custom CNN that achieves competitive performance across multiple application domains, and (2) a comprehensive comparative analysis highlighting when transfer learning and deep architectures provide substantial advantages, particularly in data-constrained environments. These findings offer practical insights for deploying deep learning models in resource-limited yet high-impact real-world visual classification tasks.