Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?

Francisco Mena, Diego Arenas, Miro Miranda, Andreas Dengel

TL;DR

The paper investigates what governs the robustness of multi-source Earth Observation models when entire data sources are missing. By evaluating six state-of-the-art models across temporal and static data/tasks, it demonstrates that performance hinges on the predictive task, data-source complementarity, and model design, with some models unexpectedly benefiting from removing a source. It introduces complementary metrics and a rigorous comparison framework to reveal that no single model is universally optimal; robustness is context-dependent and sometimes favors fewer data sources or specialized configurations. These findings suggest data-source selection and task-adaptive modeling as practical directions to streamline EO applications without sacrificing accuracy.

Abstract

In recent years, the development of robust multi-source models has emerged in the Earth Observation (EO) field. These are models that leverage data from diverse sources to improve predictive accuracy when there is missing data. Despite these advancements, the factors influencing the varying effectiveness of such models remain poorly understood. In this study, we evaluate the predictive performance of six state-of-the-art multi-source models in predicting scenarios where either a single data source is missing or only a single source is available. Our analysis reveals that the efficacy of these models is intricately tied to the nature of the task, the complementarity among data sources, and the model design. Surprisingly, we observe instances where the removal of certain data sources leads to improved predictive performance, challenging the assumption that incorporating all available data is always beneficial. These findings prompt critical reflections on model complexity and the necessity of all collected data sources, potentially shaping the way for more streamlined approaches in EO applications.

On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?

TL;DR

The paper investigates what governs the robustness of multi-source Earth Observation models when entire data sources are missing. By evaluating six state-of-the-art models across temporal and static data/tasks, it demonstrates that performance hinges on the predictive task, data-source complementarity, and model design, with some models unexpectedly benefiting from removing a source. It introduces complementary metrics and a rigorous comparison framework to reveal that no single model is universally optimal; robustness is context-dependent and sometimes favors fewer data sources or specialized configurations. These findings suggest data-source selection and task-adaptive modeling as practical directions to streamline EO applications without sacrificing accuracy.

Abstract

In recent years, the development of robust multi-source models has emerged in the Earth Observation (EO) field. These are models that leverage data from diverse sources to improve predictive accuracy when there is missing data. Despite these advancements, the factors influencing the varying effectiveness of such models remain poorly understood. In this study, we evaluate the predictive performance of six state-of-the-art multi-source models in predicting scenarios where either a single data source is missing or only a single source is available. Our analysis reveals that the efficacy of these models is intricately tied to the nature of the task, the complementarity among data sources, and the model design. Surprisingly, we observe instances where the removal of certain data sources leads to improved predictive performance, challenging the assumption that incorporating all available data is always beneficial. These findings prompt critical reflections on model complexity and the necessity of all collected data sources, potentially shaping the way for more streamlined approaches in EO applications.

Paper Structure

This paper contains 15 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Missing data in the Earth observation field
  4. How to handle source-wise missing data?
  5. How to be robust to source-wise missing data?
  6. Evaluation
  7. Datasets
  8. Compared multi-source models
  9. Implementation
  10. Results
  11. Discussion
  12. Does the individual-source performance relate to their complementary effectiveness in the task?
  13. Are additional data sources beneficial in the modeling?
  14. Is a robust model effective for all cases of source-wise missing data?
  15. Conclusion

Figures (1)

  • Figure 1: Types of missing data in the field. For the temporal missing, two cases are shown (spatial and feature wise).