Bio-inspired fine-tuning for selective transfer learning in image classification

TL;DR

BioTune tackles data-scarce transfer learning by automatically choosing which CNN layers to fine-tune and by how much, using an evolutionary search with population momentum. It introduces a per-block learning-rate scheme and a freezing threshold, optimized via stratified data partitions to reduce cost while maintaining robustness. Across nine datasets and four CNN architectures, BioTune achieves superior or competitive accuracy and efficiency compared with AutoRGN, LoRA, and other fine-tuning baselines, with notable gains on fine-grained and domain-shifted tasks. The method is reproducible (public code) and demonstrates clear practical impact for domain-specific image classification, including medical imaging scenarios.

Abstract

Deep learning has significantly advanced image analysis across diverse domains but often depends on large, annotated datasets for success. Transfer learning addresses this challenge by utilizing pre-trained models to tackle new tasks with limited labeled data. However, discrepancies between source and target domains can hinder effective transfer learning. We introduce BioTune, a novel adaptive fine-tuning technique utilizing evolutionary optimization. BioTune enhances transfer learning by optimally choosing which layers to freeze and adjusting learning rates for unfrozen layers. Through extensive evaluation on nine image classification datasets, spanning natural and specialized domains such as medical imaging, BioTune demonstrates superior accuracy and efficiency over state-of-the-art fine-tuning methods, including AutoRGN and LoRA, highlighting its adaptability to various data characteristics and distribution changes. Additionally, BioTune consistently achieves top performance across four different CNN architectures, underscoring its flexibility. Ablation studies provide valuable insights into the impact of BioTune's key components on overall performance. The source code is available at https://github.com/davilac/BioTune.

Figures (11)

• Figure 1: Overview of the transfer learning process via fine-tuning, illustrating the adaptation of a pre-trained model to a target domain.
• Figure 2: Overview of BioTune.
• Figure 3: Overview of the ResNet-50 architecture, divided into six blocks used for fine-tuning and analysis.
• Figure 4: Optimal fine-tuning configurations discovered by BioTune across datasets. Frozen layers are marked with a snowflake symbol, and learning rate weights are shown as a heatmap (0.1 to 10).
• Figure 5: Network architectures and blocks.