GoPrune: Accelerated Structured Pruning with $\ell_{2,p}$-Norm Optimization
Li Xu, Xianchao Xiu
TL;DR
GoPrune addresses the growing storage and compute demands of CNNs by enforcing channel-level sparsity through a structured regularizer $\|W\|_{2,p}^p$ with $p\in[0,1)$. It employs proximal alternating minimization to decouple the optimization into a W-update via SGD and a per-channel proximal U-update with closed-form solutions, enabling fast, one-shot pruning followed by fine-tuning. Empirical results on CIFAR-10/100 with ResNet and VGG show GoPrune achieving competitive or better accuracy than ADMM-based unstructured pruning while dramatically reducing compression time. The work demonstrates significant practical benefits for hardware-friendly pruning and suggests potential extension to large language models.
Abstract
Convolutional neural networks (CNNs) suffer from rapidly increasing storage and computational costs as their depth grows, which severely hinders their deployment on resource-constrained edge devices. Pruning is a practical approach for network compression, among which structured pruning is the most effective for inference acceleration. Although existing work has applied the $\ell_p$-norm to pruning, it only considers unstructured pruning with $p\in (0, 1)$ and has low computational efficiency. To overcome these limitations, we propose an accelerated structured pruning method called GoPrune. Our method employs the $\ell_{2,p}$-norm for sparse network learning, where the value of $p$ is extended to $[0, 1)$. Moreover, we develop an efficient optimization algorithm based on the proximal alternating minimization (PAM), and the resulting subproblems enjoy closed-form solutions, thus improving compression efficiency. Experiments on the CIFAR datasets using ResNet and VGG models demonstrate the superior performance of the proposed method in network pruning. Our code is available at https://github.com/xianchaoxiu/GoPrune.