Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

TL;DR

Robustness to missing sensors in multi-sensor earth observation time-series is a practical challenge for deploying EO models in real-world conditions. The authors introduce Input Sensor Dropout (ISensD), which randomly masks entire sensor inputs during training using a Bernoulli mask $d_s^{(i)} \sim \text{Bernoulli}(r)$ or a no-ratio variant enumerating $2^{|\mathcal{S}|}-1$ combinations, and Ensemble Sensor Invariant (ESensI), which leverages sensor-dedicated encoders with a shared prediction head and learnable sensor encodings $\rho_s$ to produce sensor-invariant predictions. Experiments on three multi-sensor temporal EO datasets show that ensemble-based approaches deliver the strongest robustness to missing sensors, with ISensD providing additional robustness gains in several settings; however, full-sensor predictive performance can be dataset- and configuration-dependent. The work offers practically relevant strategies for maintaining reliable predictions when sensor data are intermittently unavailable and motivates further analysis to identify the precise sources of robustness in such multi-sensor systems.

Abstract

Multi-sensor ML models for EO aim to enhance prediction accuracy by integrating data from various sources. However, the presence of missing data poses a significant challenge, particularly in non-persistent sensors that can be affected by external factors. Existing literature has explored strategies like temporal dropout and sensor-invariant models to address the generalization to missing data issues. Inspired by these works, we study two novel methods tailored for multi-sensor scenarios, namely Input Sensor Dropout (ISensD) and Ensemble Sensor Invariant (ESensI). Through experimentation on three multi-sensor temporal EO datasets, we demonstrate that these methods effectively increase the robustness of model predictions to missing sensors. Particularly, we focus on how the predictive performance of models drops when sensors are missing at different levels. We observe that ensemble multi-sensor models are the most robust to the lack of sensors. In addition, the sensor dropout component in ISensD shows promising robustness results.

Figures (12)

• Figure 1: Illustration of the isd method. We show three batch examples and the associated randomly masked sensors based on the sd technique.
• Figure 2: Illustration of the sensei method. The main component is the shared prediction head, which makes the ensemble-based a sensor-invariant model.
• Figure 3: of methods at different levels of missing sensors in the data.
• Figure 4: of methods at different levels of missing sensors in the data.
• Figure 5: of methods at different levels of missing sensors in the data.