EvoPruneDeepTL: An Evolutionary Pruning Model for Transfer Learning based Deep Neural Networks

TL;DR

EvoPruneDeepTL addresses the challenge of optimizing transfer-learned neural networks under limited data by pruning the last fully-connected layers through a genetic algorithm that learns sparse connections or neuron subsets. The approach alternates between two encoding schemes (neurons or connections) and jointly supports pruning and feature selection, all while freezing the convolutional feature extractor to preserve TL benefits. Across six diverse datasets and multiple feature extractors, EvoPruneDeepTL generally outperforms non-pruned baselines and standard CNN pruning methods, with feature selection frequently delivering the best accuracy and sparsity. The work demonstrates robustness via CKA-based analyses and shows practical impact by reducing final-layer complexity without sacrificing—and often improving—predictive performance, making TL-based pruning more efficient and adaptable.

Abstract

In recent years, Deep Learning models have shown a great performance in complex optimization problems. They generally require large training datasets, which is a limitation in most practical cases. Transfer learning allows importing the first layers of a pre-trained architecture and connecting them to fully-connected layers to adapt them to a new problem. Consequently, the configuration of the these layers becomes crucial for the performance of the model. Unfortunately, the optimization of these models is usually a computationally demanding task. One strategy to optimize Deep Learning models is the pruning scheme. Pruning methods are focused on reducing the complexity of the network, assuming an expected performance penalty of the model once pruned. However, the pruning could potentially be used to improve the performance, using an optimization algorithm to identify and eventually remove unnecessary connections among neurons. This work proposes EvoPruneDeepTL, an evolutionary pruning model for Transfer Learning based Deep Neural Networks which replaces the last fully-connected layers with sparse layers optimized by a genetic algorithm. Depending on its solution encoding strategy, our proposed model can either perform optimized pruning or feature selection over the densely connected part of the neural network. We carry out different experiments with several datasets to assess the benefits of our proposal. Results show the contribution of EvoPruneDeepTL and feature selection to the overall computational efficiency of the network as a result of the optimization process. In particular, the accuracy is improved, reducing at the same time the number of active neurons in the final layers.

Figures (6)

• Figure 1: Representation of both architectures
• Figure 2: Diagram of EvoPruneDeepTL.
• Figure 3: Representation of encoding strategies
• Figure 4: Visualization of pruning architectures with one layer
• Figure 5: Visualization of pruning architectures with two layers