Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Parameter and Parameterization Inference with Uncertainty Quantification through Differentiable Programming

Yongquan Qu, Mohamed Aziz Bhouri, Pierre Gentine

TL;DR

The paper addresses the challenge of closing coarse-grained PDEs for weather, climate, and turbulence by jointly inferring physical parameters and neural-network closures with quantified uncertainty. It proposes a differentiable-programming framework that combines online deterministic training with SG-HMC-based Bayesian inference to learn $(\boldsymbol{\theta}_{phy},\boldsymbol{\theta}_{NN})$ in the context of a differentiable solver $\\mathcal{M}$, yielding predictive posteriors $p(\\boldsymbol{\theta},\\gamma,\\lambda|\\mathcal{D})$. Applied to a two-layer quasi-geostrophic model, the method achieves accurate recovery of physical parameters (e.g., relative errors around a few percent) and improved forecast accuracy with credible uncertainty bands, outperforming no-parameterization and Smagorinsky baselines. The approach demonstrates a scalable path to uncertainty-aware hybrid physics-ML models for Earth system modeling and turbulence simulations, with potential extensions to larger-scale models and state inference.

Abstract

Accurate representations of unknown and sub-grid physical processes through parameterizations (or closure) in numerical simulations with quantified uncertainty are critical for resolving the coarse-grained partial differential equations that govern many problems ranging from weather and climate prediction to turbulence simulations. Recent advances have seen machine learning (ML) increasingly applied to model these subgrid processes, resulting in the development of hybrid physics-ML models through the integration with numerical solvers. In this work, we introduce a novel framework for the joint estimation of physical parameters and machine learning parameterizations with uncertainty quantification. Our framework incorporates online training and efficient Bayesian inference within a high-dimensional parameter space, facilitated by differentiable programming. This proof of concept underscores the substantial potential of differentiable programming in synergistically combining machine learning with differential equations, thereby enhancing the capabilities of hybrid physics-ML modeling.

Joint Parameter and Parameterization Inference with Uncertainty Quantification through Differentiable Programming

TL;DR

The paper addresses the challenge of closing coarse-grained PDEs for weather, climate, and turbulence by jointly inferring physical parameters and neural-network closures with quantified uncertainty. It proposes a differentiable-programming framework that combines online deterministic training with SG-HMC-based Bayesian inference to learn in the context of a differentiable solver , yielding predictive posteriors . Applied to a two-layer quasi-geostrophic model, the method achieves accurate recovery of physical parameters (e.g., relative errors around a few percent) and improved forecast accuracy with credible uncertainty bands, outperforming no-parameterization and Smagorinsky baselines. The approach demonstrates a scalable path to uncertainty-aware hybrid physics-ML models for Earth system modeling and turbulence simulations, with potential extensions to larger-scale models and state inference.

Abstract

Accurate representations of unknown and sub-grid physical processes through parameterizations (or closure) in numerical simulations with quantified uncertainty are critical for resolving the coarse-grained partial differential equations that govern many problems ranging from weather and climate prediction to turbulence simulations. Recent advances have seen machine learning (ML) increasingly applied to model these subgrid processes, resulting in the development of hybrid physics-ML models through the integration with numerical solvers. In this work, we introduce a novel framework for the joint estimation of physical parameters and machine learning parameterizations with uncertainty quantification. Our framework incorporates online training and efficient Bayesian inference within a high-dimensional parameter space, facilitated by differentiable programming. This proof of concept underscores the substantial potential of differentiable programming in synergistically combining machine learning with differential equations, thereby enhancing the capabilities of hybrid physics-ML modeling.
Paper Structure (10 sections, 15 equations, 4 figures, 2 tables)

This paper contains 10 sections, 15 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Online deterministic training:
  4. Bayesian inference and uncertainty propagation:
  5. Experiments and Results
  6. Conclusion and Discussion
  7. QG model parameters and data generation
  8. Priors and Posterior Statistics
  9. Online Deterministic training and Bayesian Inference details
  10. Additional Results

Figures (4)

  • Figure 1: Metrics evaluating the performance of online predictions over 1 year period.
  • Figure 2: Curves for training loss, validation loss, $U_1$ and $\delta$. Note that the values are averaged over all the batches within an epoch. After 50 epoches, $U_1$ and $\delta$ are fixed.
  • Figure 3: Metrics evaluating the performance of online predictions over 1 year period. Same as Figure 2, but evaluated on another test case.
  • Figure 4: Metrics evaluating the performance of online predictions over 1 year period. Same as Figure 2 and 3, but evaluated on another test case.