Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAIN: Reinforcement Algorithms for Improving Numerical Weather and Climate Models

Pritthijit Nath, Henry Moss, Emily Shuckburgh, Mark Webb

TL;DR

This study investigates reinforcement learning to improve climate model parameterisations in idealised settings. It frames parameterisation as a control task and evaluates eight continuous-action, model-free RL algorithms on two environments: SimpleClimateBiasCorrectionEnv and RadiativeConvectiveModelEnv built with climlab. Key findings show off-policy methods (DDPG, TD3, TQC) excel in bias correction, while on-policy methods (PPO, TRPO) perform best in the RCE environment; the RL-assisted RCE model achieves substantial improvements in mid-to-upper tropospheric temperature tracking. The work demonstrates RL as a viable path to physically consistent, scalable parameterisations in climate models and establishes a framework for broader integration with operational models.

Abstract

This study explores integrating reinforcement learning (RL) with idealised climate models to address key parameterisation challenges in climate science. Current climate models rely on complex mathematical parameterisations to represent sub-grid scale processes, which can introduce substantial uncertainties. RL offers capabilities to enhance these parameterisation schemes, including direct interaction, handling sparse or delayed feedback, continuous online learning, and long-term optimisation. We evaluate the performance of eight RL algorithms on two idealised environments: one for temperature bias correction, another for radiative-convective equilibrium (RCE) imitating real-world computational constraints. Results show different RL approaches excel in different climate scenarios with exploration algorithms performing better in bias correction, while exploitation algorithms proving more effective for RCE. These findings support the potential of RL-based parameterisation schemes to be integrated into global climate models, improving accuracy and efficiency in capturing complex climate dynamics. Overall, this work represents an important first step towards leveraging RL to enhance climate model accuracy, critical for improving climate understanding and predictions. Code accessible at https://github.com/p3jitnath/climate-rl.

RAIN: Reinforcement Algorithms for Improving Numerical Weather and Climate Models

TL;DR

This study investigates reinforcement learning to improve climate model parameterisations in idealised settings. It frames parameterisation as a control task and evaluates eight continuous-action, model-free RL algorithms on two environments: SimpleClimateBiasCorrectionEnv and RadiativeConvectiveModelEnv built with climlab. Key findings show off-policy methods (DDPG, TD3, TQC) excel in bias correction, while on-policy methods (PPO, TRPO) perform best in the RCE environment; the RL-assisted RCE model achieves substantial improvements in mid-to-upper tropospheric temperature tracking. The work demonstrates RL as a viable path to physically consistent, scalable parameterisations in climate models and establishes a framework for broader integration with operational models.

Abstract

This study explores integrating reinforcement learning (RL) with idealised climate models to address key parameterisation challenges in climate science. Current climate models rely on complex mathematical parameterisations to represent sub-grid scale processes, which can introduce substantial uncertainties. RL offers capabilities to enhance these parameterisation schemes, including direct interaction, handling sparse or delayed feedback, continuous online learning, and long-term optimisation. We evaluate the performance of eight RL algorithms on two idealised environments: one for temperature bias correction, another for radiative-convective equilibrium (RCE) imitating real-world computational constraints. Results show different RL approaches excel in different climate scenarios with exploration algorithms performing better in bias correction, while exploitation algorithms proving more effective for RCE. These findings support the potential of RL-based parameterisation schemes to be integrated into global climate models, improving accuracy and efficiency in capturing complex climate dynamics. Overall, this work represents an important first step towards leveraging RL to enhance climate model accuracy, critical for improving climate understanding and predictions. Code accessible at https://github.com/p3jitnath/climate-rl.
Paper Structure (29 sections, 2 equations, 9 figures, 8 tables, 8 algorithms)

This paper contains 29 sections, 2 equations, 9 figures, 8 tables, 8 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Background
  4. Advantages over traditional ML models
  5. Radiative-Convective Equilibrium
  6. RL Environments
  7. SimpleClimateBiasCorrectionEnv
  8. RadiativeConvectiveModelEnv
  9. Results
  10. Conclusion
  11. Additional Background
  12. RL Algorithm Summaries
  13. RL Environments
  14. Hyperparameter Tuning Setup
  15. Evaluation
  16. ...and 14 more sections

Figures (9)

  • Figure 1: Snapshot of the temperature profile and the differences at 100 hPa and 200 hPa in the last episodic timestep of an RL-assisted RCE integration run using DPG in rce-v0-optim-L-10k.
  • Figure A.1: (a) Thermometer (with 25 K marking intervals) visualising current temperature (indicated by red). Black line in the middle indicates the desired observed temperature of 321.75 K. (b) Line plot describing the state (temperature) evolution of both the RL agent and the biased physics model over 200 steps in one episode.
  • Figure A.2: Tephigrams displaying three different temperature profiles: RCE Model with RL (blue), RCE Model (orange), and Observations (black).
  • Figure A.3: Flow diagrams describing (a) the resource allocation in a Ray node and (b) the arrangement of Ray nodes in JASMIN
  • Figure B.4: Flow diagram describing the end-to-end result generation process for each experiment
  • ...and 4 more figures