Real-Time Compressed Sensing for Joint Hyperspectral Image Transmission and Restoration for CubeSat

TL;DR

The paper tackles real-time hyperspectral image reconstruction on CubeSats, where stripe artifacts and limited on-board resources hinder traditional methods. It introduces RTCS, a lightweight framework featuring a single linear encoder (compatible with integer-8 hardware) and a two-stream CSF-Net decoder that operates efficiently on edge devices, guided by a SAM-aware loss and stripe-masking augmentation. Key contributions include a hardware-friendly 9×1 convolutional encoder, IMFA-Net and FRDB-Net branches with grouped convolution, a novel SAM loss, and few-shot training capability that enables strong performance with small datasets. Experimental results on AVIRIS data show RTCS achieving state-of-the-art PSNR, RMSE, and SAM at very low sampling rates, while delivering real-time decoding on embedded platforms and robust restoration under stripe and transmission-noise scenarios. Overall, RTCS enables on-board sensing and rapid ground-station reconstruction, scalable to growing CubeSat fleets and stripe-affected HSI data streams.

Abstract

This paper addresses the challenges associated with hyperspectral image (HSI) reconstruction from miniaturized satellites, which often suffer from stripe effects and are computationally resource-limited. We propose a Real-Time Compressed Sensing (RTCS) network designed to be lightweight and require only relatively few training samples for efficient and robust HSI reconstruction in the presence of the stripe effect and under noisy transmission conditions. The RTCS network features a simplified architecture that reduces the required training samples and allows for easy implementation on integer-8-based encoders, facilitating rapid compressed sensing for stripe-like HSI, which exactly matches the moderate design of miniaturized satellites on push broom scanning mechanism. This contrasts optimization-based models that demand high-precision floating-point operations, making them difficult to deploy on edge devices. Our encoder employs an integer-8-compatible linear projection for stripe-like HSI data transmission, ensuring real-time compressed sensing. Furthermore, based on the novel two-streamed architecture, an efficient HSI restoration decoder is proposed for the receiver side, allowing for edge-device reconstruction without needing a sophisticated central server. This is particularly crucial as an increasing number of miniaturized satellites necessitates significant computing resources on the ground station. Extensive experiments validate the superior performance of our approach, offering new and vital capabilities for existing miniaturized satellite systems.

Figures (10)

• Figure 1: The performance and computational complexity (decoding and encoding) comparison between the proposed RTCS Network and other peer methods in terms of frame-per-second (FPS) and Spectral angle mapper (SAM), Peak-Signal-to-Noise (PSNR), and Root-Mean-Squared-Error (RMSE), where the DCSN dcsn and our RTCS are evaluated in Jetson TX2, the other methods are evaluated on a personal computer, and the sampling rate is ranged in from 0.5% to 5%.
• Figure 2: Overview of the proposed RTCS framework for hyperspectral data processing in CubeSat platforms, illustrating the compression and restoration pipeline. The process includes stripe-like sensing, encoding on a low-power chip, and efficient decoding on edge devices. The system is designed for real-time operation, emphasizing minimizing memory usage and computational load for resource-constrained satellite systems.
• Figure 3: The diagram of convolutional operation as matrix multiplication for HCS in our RTCS encoder.
• Figure 4: The overview of the proposed hardware-friendly and effective decoder for HSI compressed sensing and restoration.
• Figure 5: Network architecture of the proposed two-streamed CSF-Net, where k(c)n(c) represents the number of kernels and feature maps respectively.