A Multi-objective Complex Network Pruning Framework Based on Divide-and-conquer and Global Performance Impairment Ranking

TL;DR

The paper tackles the challenge of expensive evolutionary multi-objective pruning in large deep networks by introducing EMO-DIR, a divide-and-conquer framework that prunes sub-networks independently and fuses results via a global performance impairment ranking. It introduces cross-network constraints to keep sub-networks compatible and employs GPIR to select joint pruning schemes that minimize global performance impairment while achieving high compression. Key contributions include (i) a divide-and-conquer EMO pruning strategy that reduces the optimization space and verification cost, (ii) a cross-network constraint training regime leveraging feature detectors, and (iii) a GPIR-based method to assemble a coherent full-network pruning plan from sub-network Pareto fronts. Empirical results on CIFAR-10/100 and ImageNet-100/1K across ResNet and VGG architectures show EMO-DIR achieves higher pruning rates with comparable or slightly increased accuracy losses compared to state-of-the-art baselines, and it runs more efficiently than a whole-network EMO approach. The framework offers a scalable approach to deploying compact DNNs on resource-constrained devices without substantial performance degradation.

Abstract

Model compression plays a vital role in the practical deployment of deep neural networks (DNNs), and evolutionary multi-objective (EMO) pruning is an essential tool in balancing the compression rate and performance of the DNNs. However, due to its population-based nature, EMO pruning suffers from the complex optimization space and the resource-intensive structure verification process, especially in complex networks. To this end, a multi-objective complex network pruning framework based on divide-and-conquer and global performance impairment ranking (EMO-DIR) is proposed in this paper. Firstly, a divide-and-conquer EMO network pruning method is proposed, which decomposes the complex task of EMO pruning on the entire network into easier sub-tasks on multiple sub-networks. On the one hand, this decomposition narrows the pruning optimization space and decreases the optimization difficulty; on the other hand, the smaller network structure converges faster, so the proposed algorithm consumes lower computational resources. Secondly, a sub-network training method based on cross-network constraints is designed, which could bridge independent EMO pruning sub-tasks, allowing them to collaborate better and improving the overall performance of the pruned network. Finally, a multiple sub-networks joint pruning method based on EMO is proposed. This method combines the Pareto Fronts from EMO pruning results on multiple sub-networks through global performance impairment ranking to design a joint pruning scheme. The rich experiments on CIFAR-10/100 and ImageNet-100/1k are conducted. The proposed algorithm achieves a comparable performance with the state-of-the-art pruning methods.

Figures (9)

• Figure 1: The overall structure of the EMO-DIR.
• Figure 2: The codings of the original network structures and their pruned structures (BasicBlock, Bottleneck residual structures and VGG block VBlock).
• Figure 3: The divide-and-conquer EMO network pruning method.
• Figure 4: The sub-network training method based on cross-network constraints.
• Figure 5: The structures of feature detectors located at different positions.