Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An adjoint-free algorithm for conditional nonlinear optimal perturbations (CNOPs) via sampling

Bin Shi, Guodong Sun

TL;DR

The paper addresses computing conditional nonlinear optimal perturbations (CNOPs) without adjoint-gradient information by introducing a sampling-based, zeroth-order optimization method. It leverages a high-dimensional Stokes' theorem to express the gradient of a ball-averaged objective as an expectation and uses a sample average to approximate this gradient, with a Chernoff-type concentration bound guaranteeing probabilistic accuracy. The method is demonstrated on two models—the Burgers equation with small viscosity and the Lorenz-96 system—showing that, for comparable accuracy, the sampling approach can achieve substantial reductions in computation time relative to traditional adjoint-based or direct methods. Overall, the sampling CNOP algorithm offers a practical, scalable alternative for predictability analyses in atmospheric and oceanic contexts where adjoint models are unavailable or costly. The results indicate near-alignment with baseline methods in objective value and pattern fidelity, with clear time advantages when using a modest number of samples.

Abstract

In this paper, we propose a sampling algorithm based on state-of-the-art statistical machine learning techniques to obtain conditional nonlinear optimal perturbations (CNOPs), which is different from traditional (deterministic) optimization methods.1 Specifically, the traditional approach is unavailable in practice, which requires numerically computing the gradient (first-order information) such that the computation cost is expensive, since it needs a large number of times to run numerical models. However, the sampling approach directly reduces the gradient to the objective function value (zeroth-order information), which also avoids using the adjoint technique that is unusable for many atmosphere and ocean models and requires large amounts of storage. We show an intuitive analysis for the sampling algorithm from the law of large numbers and further present a Chernoff-type concentration inequality to rigorously characterize the degree to which the sample average probabilistically approximates the exact gradient. The experiments are implemented to obtain the CNOPs for two numerical models, the Burgers equation with small viscosity and the Lorenz-96 model. We demonstrate the CNOPs obtained with their spatial patterns, objective values, computation times, and nonlinear error growth. Compared with the performance of the three approaches, all the characters for quantifying the CNOPs are nearly consistent, while the computation time using the sampling approach with fewer samples is much shorter. In other words, the new sampling algorithm shortens the computation time to the utmost at the cost of losing little accuracy.

An adjoint-free algorithm for conditional nonlinear optimal perturbations (CNOPs) via sampling

TL;DR

The paper addresses computing conditional nonlinear optimal perturbations (CNOPs) without adjoint-gradient information by introducing a sampling-based, zeroth-order optimization method. It leverages a high-dimensional Stokes' theorem to express the gradient of a ball-averaged objective as an expectation and uses a sample average to approximate this gradient, with a Chernoff-type concentration bound guaranteeing probabilistic accuracy. The method is demonstrated on two models—the Burgers equation with small viscosity and the Lorenz-96 system—showing that, for comparable accuracy, the sampling approach can achieve substantial reductions in computation time relative to traditional adjoint-based or direct methods. Overall, the sampling CNOP algorithm offers a practical, scalable alternative for predictability analyses in atmospheric and oceanic contexts where adjoint models are unavailable or costly. The results indicate near-alignment with baseline methods in objective value and pattern fidelity, with clear time advantages when using a modest number of samples.

Abstract

In this paper, we propose a sampling algorithm based on state-of-the-art statistical machine learning techniques to obtain conditional nonlinear optimal perturbations (CNOPs), which is different from traditional (deterministic) optimization methods.1 Specifically, the traditional approach is unavailable in practice, which requires numerically computing the gradient (first-order information) such that the computation cost is expensive, since it needs a large number of times to run numerical models. However, the sampling approach directly reduces the gradient to the objective function value (zeroth-order information), which also avoids using the adjoint technique that is unusable for many atmosphere and ocean models and requires large amounts of storage. We show an intuitive analysis for the sampling algorithm from the law of large numbers and further present a Chernoff-type concentration inequality to rigorously characterize the degree to which the sample average probabilistically approximates the exact gradient. The experiments are implemented to obtain the CNOPs for two numerical models, the Burgers equation with small viscosity and the Lorenz-96 model. We demonstrate the CNOPs obtained with their spatial patterns, objective values, computation times, and nonlinear error growth. Compared with the performance of the three approaches, all the characters for quantifying the CNOPs are nearly consistent, while the computation time using the sampling approach with fewer samples is much shorter. In other words, the new sampling algorithm shortens the computation time to the utmost at the cost of losing little accuracy.
Paper Structure (8 sections, 4 theorems, 30 equations, 6 figures, 4 tables)

This paper contains 8 sections, 4 theorems, 30 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. The Basic CNOP Settings
  3. Sample-based algorithm
  4. Numerical model and experiments
  5. The Burgers equation with small viscosity
  6. The Lorenz-96 model
  7. Summary and discussion
  8. Proof of \ref{['thm: main']}

Key Result

Theorem 1

If $J$ is continuously differentiable and satisfies the gradient Lipschitz condition, i.e., for any $u_{0,1}, u_{0,2} \in B(0, \delta)$, there exists a constant $L > 0$ such that the following inequality holds as For any $t > Ld\epsilon/2$, there exists a constant $C>0$ such that the samples satisfy the concentration inequality as

Figures (6)

  • Figure 1: Spatial distributions of CNOPs (unit: $m/s$). Prediction time: on the top is $T = 30s$, and on the bottom is $T = 60s$.
  • Figure 2: Nonlinear evolution behavior of the CNOPs in terms of the norm square.
  • Figure 3: Nonlinear evolution behavior of the CNOPs in terms of the difference and relative difference of the norm square.
  • Figure 4: Spatial distributions of CNOPs.
  • Figure 5: Nonlinear evolution behavior of the CNOPs in terms of the norm square.
  • ...and 1 more figures

Theorems & Definitions (5)

  • Theorem 1
  • Lemma A.1
  • proof : Proof of Lemma \ref{['lem: gradient']}
  • Lemma A.2: Proposition 2.5.2 in vershynin2018high
  • Lemma A.3: Theorem 2.6.3 in vershynin2018high