Iterative Filter Pruning for Concatenation-based CNN Architectures

TL;DR

This work tackles pruning of concatenation-heavy CNNs used in object detection by introducing a connectivity-graph–based, automated iterative pruning pipeline. It accounts for concatenation layers via a layer connectivity graph, using sensitivity analysis and a dynamic pruning-rate scoring function to select layers and rates, followed by fine-tuning. The method achieves substantial compression (up to ~80% sparsity) with minimal accuracy loss and delivers about a 2x speedup for convolutional layers, enabling real-time FPGA performance (14 FPS for pruned YOLOv7-tiny). Evaluations on YOLOv7 and YOLOv7-tiny across COCO, FPGA, and Jetson Xavier demonstrate hardware-aware gains and provide a practical framework for pruning complex detectors. The approach is extensible to other concatenation-based architectures and offers a reproducible pathway for deployment on edge devices.

Abstract

Model compression and hardware acceleration are essential for the resource-efficient deployment of deep neural networks. Modern object detectors have highly interconnected convolutional layers with concatenations. In this work, we study how pruning can be applied to such architectures, exemplary for YOLOv7. We propose a method to handle concatenation layers, based on the connectivity graph of convolutional layers. By automating iterative sensitivity analysis, pruning, and subsequent model fine-tuning, we can significantly reduce model size both in terms of the number of parameters and FLOPs, while keeping comparable model accuracy. Finally, we deploy pruned models to FPGA and NVIDIA Jetson Xavier AGX. Pruned models demonstrate a 2x speedup for the convolutional layers in comparison to the unpruned counterparts and reach real-time capability with 14 FPS on FPGA. Our code is available at https://github.com/fzi-forschungszentrum-informatik/iterative-yolo-pruning.

Figures (9)

• Figure 1: The proposed iterative pruning approach relies on a layer connectivity graph (here exemplary for YOLOv7-tiny).
• Figure 2: Concatenation layers during pruning: two filters of the conv2 layer are pruned, so the kernels corresponding to filters should be removed from the input feature map of the conv5.
• Figure 3: Connectivity graph of an ELAN module.
• Figure 4: Pruning 50% of filters in each layer based on the smallest L2-norm before (blue) vs. after (orange) fusing the batch normalization layers into the convolutional layers.
• Figure 5: Analysis of the distribution of parameters and FLOPs over layers in unpruned YOLOv7 and YOLOv7-tiny models (a, d); pruned parameters and FLOPs and the corresponding pruning rates, selected by our algorithm (b, e); latency per layer for the models deployed to FPGA (c, f).