Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Benchmark Study of Neural Network Compression Methods for Hyperspectral Image Classification

Sai Shi

TL;DR

This study conducts a systematic evaluation of neural network compression methods for a remote sensing application, namely hyperspectral land cover classification, and examines three widely used compression strategies for convolutional neural networks: pruning, quantization, and knowledge distillation.

Abstract

Deep neural networks have achieved strong performance in image classification tasks due to their ability to learn complex patterns from high-dimensional data. However, their large computational and memory requirements often limit deployment on resource-constrained platforms such as remote sensing devices and edge systems. Network compression techniques have therefore been proposed to reduce model size and computational cost while maintaining predictive performance. In this study, we conduct a systematic evaluation of neural network compression methods for a remote sensing application, namely hyperspectral land cover classification. Specifically, we examine three widely used compression strategies for convolutional neural networks: pruning, quantization, and knowledge distillation. Experiments are conducted on two benchmark hyperspectral datasets, considering classification accuracy, memory consumption, and inference efficiency. Our results demonstrate that compressed models can significantly reduce model size and computational cost while maintaining competitive classification performance. These findings provide insights into the trade-offs between compression ratio, efficiency, and accuracy, and highlight the potential of compression techniques for enabling efficient deep learning deployment in remote sensing applications.

A Benchmark Study of Neural Network Compression Methods for Hyperspectral Image Classification

TL;DR

This study conducts a systematic evaluation of neural network compression methods for a remote sensing application, namely hyperspectral land cover classification, and examines three widely used compression strategies for convolutional neural networks: pruning, quantization, and knowledge distillation.

Abstract

Deep neural networks have achieved strong performance in image classification tasks due to their ability to learn complex patterns from high-dimensional data. However, their large computational and memory requirements often limit deployment on resource-constrained platforms such as remote sensing devices and edge systems. Network compression techniques have therefore been proposed to reduce model size and computational cost while maintaining predictive performance. In this study, we conduct a systematic evaluation of neural network compression methods for a remote sensing application, namely hyperspectral land cover classification. Specifically, we examine three widely used compression strategies for convolutional neural networks: pruning, quantization, and knowledge distillation. Experiments are conducted on two benchmark hyperspectral datasets, considering classification accuracy, memory consumption, and inference efficiency. Our results demonstrate that compressed models can significantly reduce model size and computational cost while maintaining competitive classification performance. These findings provide insights into the trade-offs between compression ratio, efficiency, and accuracy, and highlight the potential of compression techniques for enabling efficient deep learning deployment in remote sensing applications.
Paper Structure (59 sections, 5 equations, 5 figures, 13 tables)

This paper contains 59 sections, 5 equations, 5 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Datasets
  3. Raw Data
  4. Indian Pines
  5. University of Pavia
  6. Data Preprocessing
  7. Data cleaning:
  8. Data transformation:
  9. Data splitting:
  10. Approaches
  11. Convolutional Neural Network
  12. Spectral models:
  13. Spatial models:
  14. Spatial-spectral models:
  15. Network Pruning
  16. ...and 44 more sections

Figures (5)

  • Figure 1: Random selection vs. spatially disjoint samples on the IP and UP dataset, considering the same number of samples per class
  • Figure 2: The implementation details of spectral model (CNN1D) , spatial model (CNN2D), and spatial-spectral model (CNN3D)
  • Figure 3: A typical three-stage network pruning pipeline
  • Figure 4: Three types of fine-tuning schemes: (a) One-shot pruning: prune and retrain the model only once (b) Iterative pruning: prune layer by layer, and retrain the model before pruning the next layer (c) Multi-pass pruning: prune and retrain the whole network, then repeat this process until satisfied
  • Figure 5: Different distillations modes. Blue: networks are learned before distillation; Yellow: networks are learned during distillation