Generalizing across Temporal Domains with Koopman Operators

TL;DR

This work introduces Temporal Domain Generalization (TDG) and grounds it in Koopman theory, proposing Temporal Koopman Networks (TKNets) that learn a linear operator in a Koopman space to capture time-evolving domain dynamics. A key theoretical contribution is the notion of $\lambda$-consistency, which yields a bound on the target risk in terms of the forecasted versus real domain distributions, motivating distributional alignment in Koopman space. The method encodes inputs with $\phi$, maps via measurement functions $\mathcal{G}$, and learns a Koopman operator $\mathcal{K}$ to forecast the next-domain distribution $\mathcal{D}_{i+1}^{\mathcal{K}}$, training to minimize a KL-inspired loss $J$ and an inter-class distance term. Empirical results on six datasets, including RMNIST and evolving circle/plate synthetic tasks, show that TKNets consistently outperform a wide range of DG/TDG baselines, demonstrating strong extrapolation capabilities to future domains and validating the practical impact of Koopman-based TDG.

Abstract

In the field of domain generalization, the task of constructing a predictive model capable of generalizing to a target domain without access to target data remains challenging. This problem becomes further complicated when considering evolving dynamics between domains. While various approaches have been proposed to address this issue, a comprehensive understanding of the underlying generalization theory is still lacking. In this study, we contribute novel theoretic results that aligning conditional distribution leads to the reduction of generalization bounds. Our analysis serves as a key motivation for solving the Temporal Domain Generalization (TDG) problem through the application of Koopman Neural Operators, resulting in Temporal Koopman Networks (TKNets). By employing Koopman Operators, we effectively address the time-evolving distributions encountered in TDG using the principles of Koopman theory, where measurement functions are sought to establish linear transition relations between evolving domains. Through empirical evaluations conducted on synthetic and real-world datasets, we validate the effectiveness of our proposed approach.

Figures (7)

• Figure 1: (a) Evolving manner among RMNIST domains. (b) Accuracy of traditional DG methods on evolving domains. These methods cannot generalize well on outer domains ($0^\circ$ and $75^\circ$). (c) Comparison between the performance of our method and baselines on outer domains. The proposed method outperforms all the baselines.
• Figure 2: Visualization of the EvolCircle dataset. (a) 30 domains indexed by different colors, where the left bottom one is the target domain. (b) Positive and negative instances are denoted by red and blue dots respectively. (c) The decision boundaries learned by ERM. (d) Decision boundaries of the last model on all domains. (e) Decision boundaries of models in each domain.
• Figure 3: Visualization of the RPlate data set. (a) True decision boundaries evolve over domains. (b) & (c) The decision boundaries learned by ERM and TKNets on the target domain.
• Figure 4: Performance of algorithms when numbers of domains changes.
• Figure 5: Three domain index information incorporation strategies.

Theorems & Definitions (16)

• Definition 4.1: $\lambda$-Consistency
• Theorem 4.2
• Theorem 4.3
• Lemma A.1
• proof
• Lemma A.2
• Remark A.3
• proof
• Theorem A.4
• Remark A.5