Efficient CNN Compression via Multi-method Low Rank Factorization and Feature Map Similarity

TL;DR

This work tackles efficient CNN compression by reframing LRF as a multi-objective design-space exploration problem. It introduces a feature map similarity-based rank selection, a one-shot fine-tuning workflow, and a hybrid decomposition strategy that applies multiple LRF methods per layer, all integrated into TensorFlow 2.x. The approach is demonstrated on 14 CNNs across eight datasets, achieving substantial parameter and FLOPs reductions with minimal accuracy loss and outperforming VBMF and FBP in many scenarios. The results highlight the practical impact for edge- and mobile-deployable models and suggest avenues for extending the framework to other architectures and future compression techniques.

Abstract

Low-Rank Factorization (LRF) is a widely adopted technique for compressing deep neural networks (DNNs). However, it faces several challenges, including optimal rank selection, a vast design space, long fine-tuning times, and limited compatibility with different layer types and decomposition methods. This paper presents an end-to-end Design Space Exploration (DSE) methodology and framework for compressing convolutional neural networks (CNNs) that addresses all these issues. We introduce a novel rank selection strategy based on feature map similarity, which captures non-linear interactions between layer outputs more effectively than traditional weight-based approaches. Unlike prior works, our method uses a one-shot fine-tuning process, significantly reducing the overall fine-tuning time. The proposed framework is fully compatible with all types of convolutional (Conv) and fully connected (FC) layers. To further improve compression, the framework integrates three different LRF techniques for Conv layers and three for FC layers, applying them selectively on a per-layer basis. We demonstrate that combining multiple LRF methods within a single model yields better compression results than using a single method uniformly across all layers. Finally, we provide a comprehensive evaluation and comparison of the six LRF techniques, offering practical insights into their effectiveness across different scenarios. The proposed work is integrated into TensorFlow 2.x, ensuring compatibility with widely used deep learning workflows. Experimental results on 14 CNN models across eight datasets demonstrate that the proposed methodology achieves substantial compression with minimal accuracy loss, outperforming several state-of-the-art techniques.

Figures (17)

• Figure 1: Breakdown of FLOPs and parameter distribution across Conv and FC layers in various CNN architectures.
• Figure 2: Exploration space of multiple compression methods in the FLOPs-memory space. The graph at the top illustrates a Conv layer of ResNet50, while the graphs at the bottom shows a FC layer of LeNet5.
• Figure 3: Matrix decomposition using SVD (top) and QR decomposition (bottom)
• Figure 4: Tensor decomposition of Conv2D layers using Tucker (top), CP (middle), and TT (bottom) methods.
• Figure 5: LRF process for Conv2D layers using Tucker, CP, and TT decomposition (top), and for FC layers using SVD, QR, and T3F (bottom)