Climate-Invariant Machine Learning

TL;DR

The work tackles the challenge of generalizing climate-model closures under changing climates by proposing climate-invariant ML, which embeds physical transformations that stabilize input/output distributions across climates. By transforming inputs with physically motivated tools—notably $\tilde{q}_{\mathrm{RH}}(p)$, $\tilde{T}_{\mathrm{buoyancy}}(p)$, and $\tilde{\mathrm{LHF}}_{\Delta q}$—the authors train models that maintain accuracy across cold, present, and warm climates in three distinct atmospheric model configurations. Using SHAP analyses, they show these climate-invariant mappings become more local and physically interpretable, and that combining the invariance with regularization (BN/DP) improves cross-climate performance and data efficiency, especially when training data span multiple climates. The approach suggests a path toward robust, data-efficient subgrid closures for Earth system models and offers a framework for physically grounded ML that generalizes across climate regimes with practical implications for climate projections and policy-relevant simulations.

Abstract

Projecting climate change is a generalization problem: we extrapolate the recent past using physical models across past, present, and future climates. Current climate models require representations of processes that occur at scales smaller than model grid size, which have been the main source of model projection uncertainty. Recent machine learning (ML) algorithms hold promise to improve such process representations, but tend to extrapolate poorly to climate regimes they were not trained on. To get the best of the physical and statistical worlds, we propose a new framework - termed "climate-invariant" ML - incorporating knowledge of climate processes into ML algorithms, and show that it can maintain high offline accuracy across a wide range of climate conditions and configurations in three distinct atmospheric models. Our results suggest that explicitly incorporating physical knowledge into data-driven models of Earth system processes can improve their consistency, data efficiency, and generalizability across climate regimes.

Figures (9)

• Figure 1: By transforming inputs $\boldsymbol{x}$ and outputs $\boldsymbol{y}$ to match their probability density functions across climates, the algorithms can learn a transformed mapping $\widetilde{\phi}$ that holds across climates. To illustrate this, we show the marginal distributions of inputs and outputs in two different climates using blue and red lines, before (top) and after (bottom) the physical transformation.
• Figure 2: Surface temperatures in the three utilized atmospheric models. Prescribed surface temperature (in K) for (left) the aquaplanet SPCAM3 model and (right) the hypohydrostatic SAM model. (Center) Annual-mean, near-surface air temperatures in the Earth-like SPCESM2 model.
• Figure 3: Physical transformations can align distributions across climates. We show the univariate distributions of selected raw inputs $\boldsymbol{x}$: (a) 600hPa specific humidity, (b) 850hPa temperature, and (c) latent heat flux in the cold (blue), reference (gray), and warm (red) simulations of each model (SPCAM3, SPCESM2, and SAM). For each variable, we also show the PDFs of the transformed inputs $\boldsymbol{\widetilde{x}}$ as discussed in text. From top to bottom, the variables are $q$ (g/kg), RH, T (K), $B_{\mathrm{plume}}$ (m/s$^2$), LHF (W/m$^2$), and $\mathrm{LHF}_{\Delta q}$ ($\mathrm{kg\ m^{-2}s^{-1}}$). For a given variable and transformation, we use the same vertical logarithmic scale across models.
• Figure 4: All neural networks, trained in the cold climate, exhibit low error in the cold climate's test set (a), but much larger error in the warm climate's test set (b). This generalization error decreases as inputs are incrementally transformed: First no transformation (blue), then the vertical profile of specific humidity (orange), then the vertical profile of temperature (green), and finally latent heat fluxes (red). For reference, the purple line depicts an NN trained in the warm climate. We depict the tendencies' mean-squared error versus pressure, horizontally-averaged over the Tropics of SPCAM3 aquaplanet simulations, for the four model outputs: total moistening ($\dot{q}$), total heating ($\dot{T}$), longwave heating (lw) and shortwave heating (sw). Given that the raw-data NN's generalization error (blue line) greatly exceeds that of the transformed NNs, we zoom in on each panel to facilitate visualization.
• Figure 5: Model error across temperatures and configurations. Mean-Squared Error (MSE, in units W$^{2}$ m$^{-4}$) of six models trained in three simulations (first column) and evaluated over the training or validation set of the same and two other simulations (last four columns). The models (second column) are raw-data (RD) or climate-invariant (CI), multiple linear regressions (MLR) or neural nets (NN), and sometimes include dropout layers preceded by a batch normalization layer (DN). The models are trained for 20 epochs. We first provide the MSE corresponding to the epoch of minimal validation loss, then the MSE averaged over the 5 epochs with lowest validation losses (in parentheses), and finally the MSE divided by the baseline MSE, where we use the raw-data multiple linear regression as baseline. Note that "Different Temperature" refers to (+4K) for (-4K) training sets and vice versa. In each application case, we highlight the best model's error using bold font.