Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty Aware Tropical Cyclone Wind Speed Estimation from Satellite Data

Nils Lehmann, Nina Maria Gottschling, Stefan Depeweg, Eric Nalisnick

TL;DR

The paper tackles the challenge of obtaining reliable uncertainty estimates for tropical cyclone wind speed estimation from satellite data, where traditional DNNs yield only point predictions. It provides a comprehensive theoretical and empirical comparison of uncertainty quantification methods spanning deterministic, ensemble, Bayesian, Gaussian Process-based, diffusion-based approaches, and introduces a Deteministic UQ Extension (DUE) with a bi-Lipschitz constraint for improved robustness. Through extensive experiments across storm categories and wind-speed thresholds, the study demonstrates how predictive uncertainties can enhance accuracy and reveal method-specific and category-specific behavior. The work delivers practical guidance on method selection, evaluation metrics, and training strategies, contributing a robust benchmark for uncertainty-aware wind speed estimation in EO applications.

Abstract

Deep neural networks (DNNs) have been successfully applied to earth observation (EO) data and opened new research avenues. Despite the theoretical and practical advances of these techniques, DNNs are still considered black box tools and by default are designed to give point predictions. However, the majority of EO applications demand reliable uncertainty estimates that can support practitioners in critical decision making tasks. This work provides a theoretical and quantitative comparison of existing uncertainty quantification methods for DNNs applied to the task of wind speed estimation in satellite imagery of tropical cyclones. We provide a detailed evaluation of predictive uncertainty estimates from state-of-the-art uncertainty quantification (UQ) methods for DNNs. We find that predictive uncertainties can be utilized to further improve accuracy and analyze the predictive uncertainties of different methods across storm categories.

Uncertainty Aware Tropical Cyclone Wind Speed Estimation from Satellite Data

TL;DR

The paper tackles the challenge of obtaining reliable uncertainty estimates for tropical cyclone wind speed estimation from satellite data, where traditional DNNs yield only point predictions. It provides a comprehensive theoretical and empirical comparison of uncertainty quantification methods spanning deterministic, ensemble, Bayesian, Gaussian Process-based, diffusion-based approaches, and introduces a Deteministic UQ Extension (DUE) with a bi-Lipschitz constraint for improved robustness. Through extensive experiments across storm categories and wind-speed thresholds, the study demonstrates how predictive uncertainties can enhance accuracy and reveal method-specific and category-specific behavior. The work delivers practical guidance on method selection, evaluation metrics, and training strategies, contributing a robust benchmark for uncertainty-aware wind speed estimation in EO applications.

Abstract

Deep neural networks (DNNs) have been successfully applied to earth observation (EO) data and opened new research avenues. Despite the theoretical and practical advances of these techniques, DNNs are still considered black box tools and by default are designed to give point predictions. However, the majority of EO applications demand reliable uncertainty estimates that can support practitioners in critical decision making tasks. This work provides a theoretical and quantitative comparison of existing uncertainty quantification methods for DNNs applied to the task of wind speed estimation in satellite imagery of tropical cyclones. We provide a detailed evaluation of predictive uncertainty estimates from state-of-the-art uncertainty quantification (UQ) methods for DNNs. We find that predictive uncertainties can be utilized to further improve accuracy and analyze the predictive uncertainties of different methods across storm categories.
Paper Structure (16 sections, 84 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 16 sections, 84 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Additional Figures and Tables
  2. Experiments with a minimum wind speed of zero
  3. Experiments with a minimum wind speed of 34
  4. Experiments with a minimum wind speed of 64
  5. Overview of applied UQ Methods
  6. Description of UQ Methods
  7. Deterministic UQ Methods
  8. Ensemble Based UQ Methods
  9. Bayesian UQ Methods
  10. Gaussian Process Based UQ Methods
  11. Deterministic Uncertainty Estimation (DUE) - extension of DKL
  12. Quantile Based UQ Methods
  13. Diffusion Based UQ Methods
  14. Partially Stochastic Network Strategies
  15. Metrics
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Uncertainty Metrics over different storm categories.
  • Figure 2: MAE over different storm categories.
  • Figure 3: RMSE before and after selective prediction over different storm categories. The dotted bars are the RMSE after selective prediction with a threshold of 9 kts.
  • Figure 4: Uncertainty Metrics over different storm categories.
  • Figure 5: Uncertainty Metrics over different storm categories.