Backpropagation in hyperbolic chaos via adjoint shadowing

TL;DR

The paper develops an adjoint shadowing framework to extend backpropagation of long-time statistics to discrete-time and continuous-time hyperbolic chaos. It defines the adjoint shadowing operator $\mathcal{S}$ with three equivalent characterizations, derives a split-propagate expansion, and shows $\mathcal{S}(\omega)$ yields a bounded inhomogeneous adjoint solution, enabling efficient nonintrusive computation of the shadowing contribution to the linear response. It also decomposes the linear response into shadowing and unstable contributions and demonstrates continuous-time analogues, including a center-direction treatment and a well-defined shadowing/unstable decomposition. The approach yields practical, scalable algorithms for high-dimensional systems and provides a roadmap for incorporating randomness to handle non-hyperbolic regions, with potential impact in computing sensitivities of long-time statistics in chaotic dynamics. Overall, the adjoint shadowing framework unifies theory and computation for linear response in hyperbolic chaos and extends backpropagation-like methods to regimes where conventional adjoints fail.

Abstract

To generalize the backpropagation method to both discrete-time and continuous-time hyperbolic chaos, we introduce the adjoint shadowing operator $\mathcal{S}$ acting on covector fields. We show that $\mathcal{S}$ can be equivalently defined as: (a) $\mathcal{S}$ is the adjoint of the linear shadowing operator $S$; (b) $\mathcal{S}$ is given by a `split then propagate' expansion formula; (c) $\mathcal{S}(ω)$ is the only bounded inhomogeneous adjoint solution of $ω$. By (a), $\mathcal{S}$ adjointly expresses the shadowing contribution, a significant part of the linear response, where the linear response is the derivative of the long-time statistics with respect to system parameters. By (b), $\mathcal{S}$ also expresses the other part of the linear response, the unstable contribution. By (c), $\mathcal{S}$ can be efficiently computed by the nonintrusive shadowing algorithm in Ni and Talnikar (2019 J. Comput. Phys. 395 690-709), which is similar to the conventional backpropagation algorithm. For continuous-time cases, we additionally show that the linear response admits a well-defined decomposition into shadowing and unstable contributions.

Figures (2)

• Figure 1: Inhomogeneous tangent solution with zero initial condition (left), and the shadowing vector (right).
• Figure 2: Illustrations for inhomogeneous adjoint solution with zero initial condition (left), and the shadowing covector (right).

Theorems & Definitions (22)

• theorem 1: adjoint shadowing lemma for discrete-time
• theorem 2: adjoint shadowing lemma for continuous-time
• lemma 1: adjoint hyperbolicity
• Remark
• proof
• lemma 2: pathwise adjoint shadowing lemma
• proof
• theorem 2: adjoint shadowing lemma for discrete-time
• proof
• lemma 3