Loss Terms and Operator Forms of Koopman Autoencoders

TL;DR

This work systematically compares loss terms and operator representations for Koopman autoencoders, identifying robust configurations across eight differential equations spanning ODEs and PDEs. It introduces novel loss terms and evaluates dense, tridiagonal, and Jordan operator forms, highlighting the superiority of a tridiagonal operator paired with unitary or determinant losses. The study finds that the full accuracy loss is the most robust for prediction, while reconstruction and consistency losses serve well for encoding; auxiliary physics-informed terms offer selective benefits. The results provide practical guidelines for training Koopman-based neural operators and contribute to more reliable operator-learning methodologies, with code publicly available for replication and extension.

Abstract

Koopman autoencoders are a prevalent architecture in operator learning. But, the loss functions and the form of the operator vary significantly in the literature. This paper presents a fair and systemic study of these options. Furthermore, it introduces novel loss terms.

Figures (13)

• Figure 1: Discretization of the Koopman formulation into a numerical scheme: The physical states at successive time points are denoted by $s_0$, $s_1$, $\dots$, $s_{n-1}$ and $s_n$. The encoded states at successive time points are denoted by $e_0$, $e_1$, $\dots$, $e_{n-1}$ and $e_n$. The function $f$ is the true time evolution of the physical state by the time step. The discretized Koopman operator, encoder and decoder are denoted by $K$, $E$ and $R$, respectively.
• Figure 2: The mean effect of the different options of the numerical experiment for simple harmonic motion. The top left plot compares the mean effect of the different accuracy loss terms. The top right plot compares the mean effect of the different embedding loss terms. The bottom left plot compares the mean effect of the different operator loss terms. The bottom right plot compares the mean effect of whether or not a mask is used.
• Figure 3: Direct comparison of loss terms for the simple harmonic motion. On the left, the accuracy term is varied while the other terms are fixed. On the right, the embedding term is varied while the other terms are fixed. In each comparison, the embedding dimension is $64$, a mask is used, the determinant loss is used, and an auxiliary loss term is not used. In the comparison of the accuracy terms, the reconstruction loss is used. In the comparison of the embedding terms, full accuracy loss is used.
• Figure 4: The mean effect of the different options of the numerical experiment for the pendulum. The top left plot compares the mean effect of the different accuracy loss terms. The top right plot compares the mean effect of the different embedding loss terms. The center left plot compares the mean effect of the different operator loss terms. The center right plot compares the mean effect of whether or not a mask is used. The bottom left plot compares the mean effect of the different auxiliary loss terms.
• Figure 5: The mean effect of the different options of the numerical experiment for the Lorenz system. The top left plot compares the mean effect of the different accuracy loss terms. The top right plot compares the mean effect of the different embedding loss terms. The bottom left plot compares the mean effect of the different operator loss terms. The bottom right plot compares the mean effect of whether or not a mask is used.