Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Property Testing for Ocean Models. Can We Specify It? (Invited Talk)

Deepak A. Cherian

TL;DR

The paper addresses the oracle problem in ocean modeling by proposing a framework to recast geophysical fluid dynamics and ocean phenomena as property-based tests. It builds a taxonomy of dynamical properties—invariants, postconditions, metamorphic relations, inductive ideas, and model-based benchmarks—grounded in simplified GFD limits and canonical solutions. It argues that many existing ocean-model tests already embody these ideas, while providing a blueprint for formalizing physics-based properties to improve correctness checks. The work highlights practical challenges, notably transient dynamics and test-case shrinking, and suggests that integrating these tests with current regression and intercomparison practices could strengthen ocean-model verification.

Abstract

I take inspiration from the property-testing literature, particularly the work of Prof. John Hughes, and explore how such ideas might be applied to numerical models of the ocean. Specifically, I ask whether geophysical fluid dynamics (GFD) theory, expressed as property tests, might be used to address the oracle problem of testing the correctness of ocean models. I propose that a number of simple idealized GFD problems can be framed as property tests. These examples clearly illustrate how physics naturally lends itself to specifying property tests. Which of these proposed tests might be most feasible and useful, remains to be seen.

Property Testing for Ocean Models. Can We Specify It? (Invited Talk)

TL;DR

The paper addresses the oracle problem in ocean modeling by proposing a framework to recast geophysical fluid dynamics and ocean phenomena as property-based tests. It builds a taxonomy of dynamical properties—invariants, postconditions, metamorphic relations, inductive ideas, and model-based benchmarks—grounded in simplified GFD limits and canonical solutions. It argues that many existing ocean-model tests already embody these ideas, while providing a blueprint for formalizing physics-based properties to improve correctness checks. The work highlights practical challenges, notably transient dynamics and test-case shrinking, and suggests that integrating these tests with current regression and intercomparison practices could strengthen ocean-model verification.

Abstract

I take inspiration from the property-testing literature, particularly the work of Prof. John Hughes, and explore how such ideas might be applied to numerical models of the ocean. Specifically, I ask whether geophysical fluid dynamics (GFD) theory, expressed as property tests, might be used to address the oracle problem of testing the correctness of ocean models. I propose that a number of simple idealized GFD problems can be framed as property tests. These examples clearly illustrate how physics naturally lends itself to specifying property tests. Which of these proposed tests might be most feasible and useful, remains to be seen.
Paper Structure (19 sections, 2 equations, 3 figures)

This paper contains 19 sections, 2 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Testing climate models
  4. Generative property-based testing
  5. Geophysical Fluid Dynamics (GFD)
  6. Representing a rotating planet
  7. Dynamical property tests for ocean models
  8. Is there an invariant?
  9. Conservation of properties
  10. Symmetries
  11. Balanced flows remain balanced
  12. Using invariants for validity testing
  13. What is the postcondition?
  14. Resonant frequencies resonate
  15. Symmetry yields asymmetry
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Schematic of the Community Earth System Model version 3 (CESM3) lawrenceCommunityEarthSystem.
  • Figure 2: Sea surface speed from a global 1/12° simulation using the Oceananigans model Silvestri2024blog. Note several properties of the ocean circulation: (a) faster currents at western continental boundaries of ocean basins, (b) the east-west orientation of flows in the Tropics away from continental boundaries, and (c) the presence of coherent vortices (circular features).
  • Figure 3: Sea surface temperature in the Tropical Pacific from a regional MIT General Circulation Model simulation using a $1/20°\approx5\km$ spacing Cherian2021. Darker colors indicate colder water. The full domain spans 20 degrees of latitude and 80 degrees of longitude. The complex and turbulent nature of the flow is evident.