Looking into a Pixel by Nonlinear Unmixing -- A Generative Approach

Abstract

Due to the large footprint of pixels in remote sensing imagery, hyperspectral unmixing (HU) has become an important and necessary procedure in hyperspectral image analysis. Traditional HU methods rely on a prior spectral mixing model, especially for nonlinear mixtures, which has largely limited the performance and generalization capacity of the unmixing approach. In this paper, we address the challenging problem of hyperspectral nonlinear unmixing (HNU) without explicit knowledge of the mixing model. Inspired by the principle of generative models, where images of the same distribution can be generated as that of the training images without knowing the exact probability distribution function of the image, we develop an invertible mixing-unmixing process via a bi-directional GAN framework, constrained by both the cycle consistency and the linkage between linear and nonlinear mixtures. The combination of cycle consistency and linear linkage provides powerful constraints without requiring an explicit mixing model. We refer to the proposed approach as the linearly-constrained CycleGAN unmixing net, or LCGU net. Experimental results indicate that the proposed LCGU net exhibits stable and competitive performance across different datasets compared with other state-of-the-art model-based HNU methods.

Figures (4)

• Figure 1: The cycle consistency in unmixing. Left: Summary of the bi-directional data flow. Right-top: the unmixing-mixing flow in the raw image domain. Right-bottom: the mixing-unmixing flow in abundance domain.
• Figure 2: The effect of the bi-directional structure as compared to the uni-directional structure.
• Figure 3: The effect of the semantic constraint between linear and nonlinear mixtures.
• Figure 4: The effect of the reconstruction loss versus semantic loss when incorporating the semantic constraint.