Learning Structured Sparsity in Deep Neural Networks

TL;DR

This work tackles the computational burden of large deep neural networks by learning structured sparsity patterns that align with hardware memory access. It introduces Structured Sparsity Learning (SSL), a generic framework based on group Lasso that regularizes multiple network structures (filters, channels, shapes, and depth) during training to produce compact, accelerable CNNs. Experiments on MNIST, CIFAR-10, and ImageNet demonstrate substantial practical speedups on CPU and GPU with minimal or even improved accuracy, notably achieving 5.1x CPU and 3.1x GPU speedups for AlexNet and enabling depth reduction in ResNet without sacrificing performance. The approach provides a principled, plug-in method to combine structured sparsity with low-rank techniques, yielding hardware-friendly model compression that benefits real-world deployment.

Abstract

High demand for computation resources severely hinders deployment of large-scale Deep Neural Networks (DNN) in resource constrained devices. In this work, we propose a Structured Sparsity Learning (SSL) method to regularize the structures (i.e., filters, channels, filter shapes, and layer depth) of DNNs. SSL can: (1) learn a compact structure from a bigger DNN to reduce computation cost; (2) obtain a hardware-friendly structured sparsity of DNN to efficiently accelerate the DNNs evaluation. Experimental results show that SSL achieves on average 5.1x and 3.1x speedups of convolutional layer computation of AlexNet against CPU and GPU, respectively, with off-the-shelf libraries. These speedups are about twice speedups of non-structured sparsity; (3) regularize the DNN structure to improve classification accuracy. The results show that for CIFAR-10, regularization on layer depth can reduce 20 layers of a Deep Residual Network (ResNet) to 18 layers while improve the accuracy from 91.25% to 92.60%, which is still slightly higher than that of original ResNet with 32 layers. For AlexNet, structure regularization by SSL also reduces the error by around ~1%. Open source code is in https://github.com/wenwei202/caffe/tree/scnn

Figures (7)

• Figure 1: Evaluation speedups of AlexNet on GPU platforms and the sparsity. conv1 refers to convolutional layer 1, and so forth. Baseline is profiled by GEMM of cuBLAS. The sparse matrixes are stored in the format of Compressed Sparse Row (CSR) and accelerated by cuSPARSE.
• Figure 2: The proposed structured sparsity learning (SSL) for DNNs. Weights in filters are split into multiple groups. Through group Lasso regularization, a more compact DNN is obtained by removing some groups. The figure illustrates the filter-wise, channel-wise, shape-wise, and depth-wise structured sparsity that were explored in the work.
• Figure 3: Learned conv1 filters in LeNet 1 (top), LeNet 2 (middle) and LeNet 3 (bottom)
• Figure 4: (a) Results of learning the number of neurons in MLP. (b) the connection numbers of input neurons (i.e. pixels) in MLP 2 after SSL.
• Figure 5: Learned conv1 filters in ConvNet 1 (top), ConvNet 2 (middle) and ConvNet 3 (bottom)