Robustness Test for AI Forecasting of Hurricane Florence Using FourCastNetv2 and Random Perturbations of the Initial Condition

TL;DR

The paper investigates the robustness of NVIDIA FourCastNetv2 (FCNv2) for hurricane forecasting under imperfect initial conditions. It uses ERA5 data for Hurricane Florence (Sept 13–16, 2018) and conducts two robustness tests: Gaussian-noise perturbations of the initial state and fully random initial conditions, evaluating trajectory accuracy via mean trajectory error and global field consistency via MSL-pressure biases. Results show FCNv2 preserves large-scale hurricane tracks under moderate noise but underestimates intensity, with increasing trajectory errors at high noise; random inits yield smooth, coherent forecasts but can retain unphysical traits in some fields. The findings inform ensemble-based and physics-informed approaches and highlight practical considerations for deploying data-driven AI forecasts in operational settings.

Abstract

Understanding the robustness of a weather forecasting model with respect to input noise or different uncertainties is important in assessing its output reliability, particularly for extreme weather events like hurricanes. In this paper, we test sensitivity and robustness of an artificial intelligence (AI) weather forecasting model: NVIDIAs FourCastNetv2 (FCNv2). We conduct two experiments designed to assess model output under different levels of injected noise in the models initial condition. First, we perturb the initial condition of Hurricane Florence from the European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis v5 (ERA5) dataset (September 13-16, 2018) with varying amounts of Gaussian noise and examine the impact on predicted trajectories and forecasted storm intensity. Second, we start FCNv2 with fully random initial conditions and observe how the model responds to nonsensical inputs. Our results indicate that FCNv2 accurately preserves hurricane features under low to moderate noise injection. Even under high levels of noise, the model maintains the general storm trajectory and structure, although positional accuracy begins to degrade. FCNv2 consistently underestimates storm intensity and persistence across all levels of injected noise. With full random initial conditions, the model generates smooth and cohesive forecasts after a few timesteps, implying the models tendency towards stable, smoothed outputs. Our approach is simple and portable to other data-driven AI weather forecasting models.

Figures (12)

• Figure 1: The wind velocity field of Hurricane Florence at 18:00 UTC, September 13, 2018: Wind speed (depicted by the color bar) and direction (depicted by the arrows) for the central Atlantic area (30°N–40°N latitude, 70°W–90°W longitude) over all 13 pressure levels.
• Figure 2: Trajectories of Hurricane Florence (September 13–17, 2018): Truth in blue, and forecast in red from FCNv2 predictions under varying levels of Gaussian noise applied to the initial condition: (a) 0% noise added, (b) 5% noise added, (c) 20% noise added, and (d) 50% noise added. Noise percentages and application procedure are described in Section 2c. Each of the shown predictions are the median outcome of 30 runs of each noise level in terms of cumulative positional error, explained in Section 2d.
• Figure 3: Perturbed initial conditions for mean sea level pressure (MSL) with varying levels of noise added: (a) Original ERA5 data (0% noise added), (b) 5% noise added, (c) 20% noise added, and (d) 50% noise added.
• Figure 4: Fully random initial conditions generated from different distributions: (a) $\chi^2$, (b) lognormal, (c) normal, and (d) uniform. The fields appear primarily green because most sampled values are near the center of the color bar. Although this particular color bar is for MSL, these fully random initial conditions can be generated for any of the 73 variables.
• Figure 5: Mean trajectory error ($\times$ 100 km) as a function of noise level (%) with the vertical bars indicating the standard deviation from 30 trials.