Artificial intelligence to advance Earth observation: : A review of models, recent trends, and pathways forward

TL;DR

This paper proposes to review the thriving ecosystem focusing on developing AI models for Earth observation, its recent trends, and sketch potential pathways for future advances to fill the knowledge gap.

Abstract

Earth observation (EO) is a prime instrument for monitoring land and ocean processes, studying the dynamics at work, and taking the pulse of our planet. This article gives a bird's eye view of the essential scientific tools and approaches informing and supporting the transition from raw EO data to usable EO-based information. The promises, as well as the current challenges of these developments, are highlighted under dedicated sections. Specifically, we cover the impact of (i) Computer vision; (ii) Machine learning; (iii) Advanced processing and computing; (iv) Knowledge-based AI; (v) Explainable AI and causal inference; (vi) Physics-aware models; (vii) User-centric approaches; and (viii) the much-needed discussion of ethical and societal issues related to the massive use of ML technologies in EO.

Figures (7)

• Figure 1: Conceptual overview of this perspective paper: (a) different levels of algorithms emerge from the areas of machine learning (ML) and interact with computer vision (CV), computer science, and statistics to learn patterns and associations from observational data. The models must integrate domain knowledge and biogeophysical constraints to advance in the modeling and understanding the Earth's processes. The ultimate goal is to provide intelligent, interoperable, actionable, trustworthy, robust systems whose decisions should be accountable. (b) The field of AI-- and specifically, the area of ML within it -- interacts (and is embedded into) several systems to realise such ambitious goals, from high-performance computing platforms in digital twins to Earth system model simulations and products, a wide range of Earth observation data, the characterisation and quantification of uncertainty, and the (active) role of the users. (c) The processing chain in AI goes from modeling (e.g. classification, detection, parameter retrieval) with ML, CV and high-performance computing techniques that answer 'what questions', to the more ambitious goals of explainable AI, causal relations and ontologies that answer 'what if' questions, and finally to communicate decisions, which involves ethical issues, trust and interaction with the user.
• Figure 2: Different learning paradigms are relevant for AI applications in Earth observation, each with their own requirements and challenges. See text for details.
• Figure 3: Pre-trained transformer models for Earth observation are typically trained via the masked auto-encoder (MAE) principle. Image from satmae2022 (© 2022, BY-SA).
• Figure 4: An Actor model-based EO ecosystem architecture. Image from Wall2021 (© 2021, BY-SA).
• Figure 5: Methodologies for understanding complex systems such as the Earth. Top: Ontologies can represent both symbolic and numeric knowledge, reason only based on cognitive semantics and share knowledge on the interpretation of data, e.g. remote sensing images. Following arvor2019ontologies, we can describe complex concepts such as "forest" as a particular 'Entity' with some attributes or properties (e.g. NPP and NDVI levels) which can have associated values (e.g. high or low). This object can also be defined according to many intertwined properties and value levels. Middle: Many XAI methods are available to obtain explanations from ML models, typically categorised in feature attribution methods, explainable by design, distillation-based and contrastive ras2022explainable, which fall under post-hoc methods (indicated by the orange dot), local (indicated by the yellow dot) or model-agnostic (indicated by the red dot). However, XAI only explains what the ML model learned, which can be correct for wrong reasons. Causality advances in the interpretability rudin2019stopPearl2000, and proposes steps beyond the mere association rules developed in conventional ML -that can only answer 'what' questions- by advancing in discovering causal structures in the data, answering 'how' (intervention analysis) and 'why' (counterfactuals) queries. Bottom: ML and domain knowledge can interact in many ways camps2018physics: from inverting or emulating complex codes (e.g. RTM or climate models) to the estimation of parameters constrained by physics-based losses or ML-model coupling in hybrid modelling that allows estimating unobserved parameters Reichstein19nat; and to the direct discovery of fully interpretable equations and laws from data CampsValls23physcausaldiscovery.