Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compression and Acceleration of Neural Networks for Communications

Jiajia Guo, Jinghe Wang, Chao-Kai Wen, Shi Jin, Geoffrey Ye Li

TL;DR

This article investigates how to compress and accelerate the neural networks (NNs) in communication systems and identifies some challenges on NN compression and acceleration in DL-based communications.

Abstract

Deep learning (DL) has achieved great success in signal processing and communications and has become a promising technology for future wireless communications. Existing works mainly focus on exploiting DL to improve the performance of communication systems. However, the high memory requirement and computational complexity constitute a major hurdle for the practical deployment of DL-based communications. In this article, we investigate how to compress and accelerate the neural networks (NNs) in communication systems. After introducing the deployment challenges for DL-based communication algorithms, we discuss some representative NN compression and acceleration techniques. Afterwards, two case studies for multiple-input-multiple-output (MIMO) communications, including DL-based channel state information feedback and signal detection, are presented to show the feasibility and potential of these techniques. We finally identify some challenges on NN compression and acceleration in DL-based communications and provide a guideline for subsequent research.

Compression and Acceleration of Neural Networks for Communications

TL;DR

This article investigates how to compress and accelerate the neural networks (NNs) in communication systems and identifies some challenges on NN compression and acceleration in DL-based communications.

Abstract

Deep learning (DL) has achieved great success in signal processing and communications and has become a promising technology for future wireless communications. Existing works mainly focus on exploiting DL to improve the performance of communication systems. However, the high memory requirement and computational complexity constitute a major hurdle for the practical deployment of DL-based communications. In this article, we investigate how to compress and accelerate the neural networks (NNs) in communication systems. After introducing the deployment challenges for DL-based communication algorithms, we discuss some representative NN compression and acceleration techniques. Afterwards, two case studies for multiple-input-multiple-output (MIMO) communications, including DL-based channel state information feedback and signal detection, are presented to show the feasibility and potential of these techniques. We finally identify some challenges on NN compression and acceleration in DL-based communications and provide a guideline for subsequent research.

Paper Structure

This paper contains 13 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Trend of Growing NN complexity
  3. NN compression and acceleration
  4. Knowledge Distillation
  5. Efficient NN Architecture Design
  6. Network Pruning
  7. Network Quantization
  8. Low-Rank Approximation
  9. Hardware Design
  10. Two Case Studies for Massive MIMO
  11. CSI Feedback
  12. Signal Detection
  13. conclusion and discussions

Figures (4)

  • Figure 1: Overview of the training and implementation strategy of DL-based communication algorithms.
  • Figure 2: Fire module architecture: a squeeze layer and a branching layer with $1\times1$ and $3\times3$ filtersiandola2016squeezenet.
  • Figure 3: NMSE performance and weight number comparison between ConvCsiNet and ConvSquCsiNet.
  • Figure 4: BER performance comparison between FullyCon and the pruned or quantized one.