Paradigm selection for Data Fusion of SAR and Multispectral Sentinel data applied to Land-Cover Classification

TL;DR

The goals are to provide a systematic procedure for choosing the best data fusion framework, resulting in the best classification results, once the basic structure for the CNN has been defined, and to help interested researchers in their work when data fusion applied to remote sensing is involved.

Abstract

Data fusion is a well-known technique, becoming more and more popular in the Artificial Intelligence for Earth Observation (AI4EO) domain mainly due to its ability of reinforcing AI4EO applications by combining multiple data sources and thus bringing better results. On the other hand, like other methods for satellite data analysis, data fusion itself is also benefiting and evolving thanks to the integration of Artificial Intelligence (AI). In this letter, four data fusion paradigms, based on Convolutional Neural Networks (CNNs), are analyzed and implemented. The goals are to provide a systematic procedure for choosing the best data fusion framework, resulting in the best classification results, once the basic structure for the CNN has been defined, and to help interested researchers in their work when data fusion applied to remote sensing is involved. The procedure has been validated for land-cover classification but it can be transferred to other cases.

Figures (5)

• Figure 1: Early Data Fusion Paradigm
• Figure 2: Joint Data Fusion Paradigm
• Figure 3: Late Data Fusion Paradigm
• Figure 4: Proposed models architecture: Basic CNN structure (top row), Early Fusion (middle row - left), Joint Fusion (middle row - right), Late Fusion Mean Strategy (bottom row - left) and Late Fusion Weighted Strategy (bottom row - right). The figure shows as an example of classification for the "river" class.
• Figure 5: Evaluation metrics: visual representation of Table \ref{['tab:metrics']}