Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks

Judy X Yang, Jing Wang, Chen Hong Sui, Zekun Long, Jun Zhou

TL;DR

This work tackles hyperspectral and LiDAR data fusion for remote sensing classification in resource-constrained environments. It introduces the Fused non-linear Space framework and the HSLiNet model, which uses bidirectional reversed networks and early fusion after patch embedding to combine spectral and spatial cues without heavy self-attention. Key contributions include the FusedBiRNet with forward and backward spectral processing, the delta-modulated non-linearity for state updates, and empirical validation on the Houston 2013 dataset showing superior OA, AA, and Kappa with reduced computational load. The results demonstrate practical impact for real-time remote sensing tasks in limited-resource settings, offering a competitive alternative to Transformer-based approaches like SpectralFormer.

Abstract

The integration of hyperspectral imaging (HSI) and LiDAR data within new linear feature spaces offers a promising solution to the challenges posed by the high-dimensionality and redundancy inherent in HSIs. This study introduces a dual linear fused space framework that capitalizes on bidirectional reversed convolutional neural network (CNN) pathways, coupled with a specialized spatial analysis block. This approach combines the computational efficiency of CNNs with the adaptability of attention mechanisms, facilitating the effective fusion of spectral and spatial information. The proposed method not only enhances data processing and classification accuracy, but also mitigates the computational burden typically associated with advanced models such as Transformers. Evaluations of the Houston 2013 dataset demonstrate that our approach surpasses existing state-of-the-art models. This advancement underscores the potential of the framework in resource-constrained environments and its significant contributions to the field of remote sensing.

HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks

TL;DR

This work tackles hyperspectral and LiDAR data fusion for remote sensing classification in resource-constrained environments. It introduces the Fused non-linear Space framework and the HSLiNet model, which uses bidirectional reversed networks and early fusion after patch embedding to combine spectral and spatial cues without heavy self-attention. Key contributions include the FusedBiRNet with forward and backward spectral processing, the delta-modulated non-linearity for state updates, and empirical validation on the Houston 2013 dataset showing superior OA, AA, and Kappa with reduced computational load. The results demonstrate practical impact for real-time remote sensing tasks in limited-resource settings, offering a competitive alternative to Transformer-based approaches like SpectralFormer.

Abstract

The integration of hyperspectral imaging (HSI) and LiDAR data within new linear feature spaces offers a promising solution to the challenges posed by the high-dimensionality and redundancy inherent in HSIs. This study introduces a dual linear fused space framework that capitalizes on bidirectional reversed convolutional neural network (CNN) pathways, coupled with a specialized spatial analysis block. This approach combines the computational efficiency of CNNs with the adaptability of attention mechanisms, facilitating the effective fusion of spectral and spatial information. The proposed method not only enhances data processing and classification accuracy, but also mitigates the computational burden typically associated with advanced models such as Transformers. Evaluations of the Houston 2013 dataset demonstrate that our approach surpasses existing state-of-the-art models. This advancement underscores the potential of the framework in resource-constrained environments and its significant contributions to the field of remote sensing.

Paper Structure

This paper contains 13 sections, 8 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. HSI-LiDAR non-linear Fusion Model Preliminaries
  4. Forward and Backward Spectral Dependencies
  5. Proposed Method Overview and Architecture Description
  6. HSI-LiDAR non-linear Fusion Block
  7. Experiments
  8. Dataset
  9. Houston 2013
  10. Implementation Details
  11. Experiment Results
  12. Ablation Study
  13. Conclusion

Figures (3)

  • Figure 1: The architectural overview of the Proposed HSI and Lidar Fusion model. The framework consists of four main components: (A) A hyperspectral image patch with dimensions $p \times p \times \text{CH}$, LiDAR patch with dimension $p \times p \times \text{1}$; (B) The BiNetworks Block for fused spectral feature and spatial feature processing; (C) Feature fusion Block; (D) The classifier. The process begins by extracting patches that serve as input to the HSI-LiDAR non-linear block. The HSI block includes LiDAR spatial processing stage and HSI unique forward and backward operations, with the concatenation of Binetworks output and LiDAR spatial feature process forming the input to the classifier block.
  • Figure 2: Visualization and classification maps for the Houston2013 dataset. Ground-truth map, six comparative methods and HSLiNet.
  • Figure 3: OA and Epoch relationship based on different Training loops