Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Energy-Efficient Artefact Detection Accelerator on FPGAs for Hyper-Spectral Satellite Imagery

Cornell Castelino, Shashwat Khandelwal, Shanker Shreejith, Sharatchandra Varma Bogaraju

TL;DR

The paper tackles onboard artefact detection for hyperspectral satellite imagery to conserve energy and downlink bandwidth. It introduces an unsupervised 2D convolutional autoencoder quantised to 8-bit and deployed on a Zynq Ultrascale+ FPGA using the DPU in Vitis-AI, trained on multiple HSI datasets. The method achieves an F1 of 92.77% with 0% FPR and a per-image energy of 21.52 mJ, while delivering a latency of 4 ms per spectral band and a throughput of 250 images/s, outperforming several state-of-the-art detectors in latency and energy. This approach enables robust, generalised artefact detection suitable for CubeSats, with practical deployment considerations via a pre-existing DPU and post-deployment model updates. The work demonstrates a viable path toward energy-efficient, onboard data selection in hyperspectral remote sensing.

Abstract

Hyper-Spectral Imaging (HSI) is a crucial technique for analysing remote sensing data acquired from Earth observation satellites. The rich spatial and spectral information obtained through HSI allows for better characterisation and exploration of the Earth's surface over traditional techniques like RGB and Multi-Spectral imaging on the downlinked image data at ground stations. Sometimes, these images do not contain meaningful information due to the presence of clouds or other artefacts, limiting their usefulness. Transmission of such artefact HSI images leads to wasteful use of already scarce energy and time costs required for communication. While detecting such artefacts before transmitting the HSI image is desirable, the computational complexity of these algorithms and the limited power budget on satellites (especially CubeSats) are key constraints. This paper presents an unsupervised learning-based convolutional autoencoder (CAE) model for artefact identification of acquired HSI images at the satellite and a deployment architecture on AMD's Zynq Ultrascale FPGAs. The model is trained and tested on widely used HSI image datasets: Indian Pines, Salinas Valley, the University of Pavia and the Kennedy Space Center. For deployment, the model is quantised to 8-bit precision, fine-tuned using the Vitis-AI framework and integrated as a subordinate accelerator using AMD's Deep-Learning Processing Units (DPU) instance on the Zynq device. Our tests show that the model can process each spectral band in an HSI image in 4 ms, 2.6x better than INT8 inference on Nvidia's Jetson platform & 1.27x better than SOTA artefact detectors. Our model also achieves an f1-score of 92.8% and FPR of 0% across the dataset, while consuming 21.52 mJ per HSI image, 3.6x better than INT8 Jetson inference & 7.5x better than SOTA artefact detectors, making it a viable architecture for deployment in CubeSats.

An Energy-Efficient Artefact Detection Accelerator on FPGAs for Hyper-Spectral Satellite Imagery

TL;DR

The paper tackles onboard artefact detection for hyperspectral satellite imagery to conserve energy and downlink bandwidth. It introduces an unsupervised 2D convolutional autoencoder quantised to 8-bit and deployed on a Zynq Ultrascale+ FPGA using the DPU in Vitis-AI, trained on multiple HSI datasets. The method achieves an F1 of 92.77% with 0% FPR and a per-image energy of 21.52 mJ, while delivering a latency of 4 ms per spectral band and a throughput of 250 images/s, outperforming several state-of-the-art detectors in latency and energy. This approach enables robust, generalised artefact detection suitable for CubeSats, with practical deployment considerations via a pre-existing DPU and post-deployment model updates. The work demonstrates a viable path toward energy-efficient, onboard data selection in hyperspectral remote sensing.

Abstract

Hyper-Spectral Imaging (HSI) is a crucial technique for analysing remote sensing data acquired from Earth observation satellites. The rich spatial and spectral information obtained through HSI allows for better characterisation and exploration of the Earth's surface over traditional techniques like RGB and Multi-Spectral imaging on the downlinked image data at ground stations. Sometimes, these images do not contain meaningful information due to the presence of clouds or other artefacts, limiting their usefulness. Transmission of such artefact HSI images leads to wasteful use of already scarce energy and time costs required for communication. While detecting such artefacts before transmitting the HSI image is desirable, the computational complexity of these algorithms and the limited power budget on satellites (especially CubeSats) are key constraints. This paper presents an unsupervised learning-based convolutional autoencoder (CAE) model for artefact identification of acquired HSI images at the satellite and a deployment architecture on AMD's Zynq Ultrascale FPGAs. The model is trained and tested on widely used HSI image datasets: Indian Pines, Salinas Valley, the University of Pavia and the Kennedy Space Center. For deployment, the model is quantised to 8-bit precision, fine-tuned using the Vitis-AI framework and integrated as a subordinate accelerator using AMD's Deep-Learning Processing Units (DPU) instance on the Zynq device. Our tests show that the model can process each spectral band in an HSI image in 4 ms, 2.6x better than INT8 inference on Nvidia's Jetson platform & 1.27x better than SOTA artefact detectors. Our model also achieves an f1-score of 92.8% and FPR of 0% across the dataset, while consuming 21.52 mJ per HSI image, 3.6x better than INT8 Jetson inference & 7.5x better than SOTA artefact detectors, making it a viable architecture for deployment in CubeSats.
Paper Structure (14 sections, 6 equations, 7 figures, 6 tables)

This paper contains 14 sections, 6 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. HSI Classification & Artefact Detection
  4. Related Works
  5. System Architecture
  6. Convolutional Auto-Encoder as an anomaly detector
  7. Design Space Exploration for CAE parameters
  8. Dataset and Training
  9. Custom Reconstruction Error after Inference
  10. Hardware Generation and Integration with the CAE Model
  11. Deployment and Experimental Results
  12. Model Performance
  13. Latency, Power and Resource Consumption
  14. Conclusion

Figures (7)

  • Figure 1: The concept behind imaging spectroscopy. An airborne or spaceborne satellite samples multiple spectral wavebands over a preset area. The hyperspectral image cube is then pre-processed with an onboard image processor before transmitting the cube image to a ground station.
  • Figure 2: Example of a frame from an HSI imagecube with a cloud artefact
  • Figure 3: The proposed model convolutional autoencoder model as a defect detection.
  • Figure 4: MLSE model loss per epoch.
  • Figure 5: Proposed system architecture of the integrated ADS. The quantised CAE model is accelerated on the PL part of the FPGA device.
  • ...and 2 more figures