Toward Physics-guided Time Series Embedding

Abstract

In various scientific and engineering fields, the primary research areas have revolved around physics-based dynamical systems modeling and data-driven time series analysis. According to the embedding theory, dynamical systems and time series can be mutually transformed using observation functions and physical reconstruction techniques. Based on this, we propose Embedding Duality Theory, where the parameterized embedding layer essentially provides a linear estimation of the non-linear time series dynamics. This theory enables us to bypass the parameterized embedding layer and directly employ physical reconstruction techniques to acquire a data embedding representation. Utilizing physical priors results in a 10X reduction in parameters, a 3X increase in speed, and maximum performance boosts of 18% in expert, 22% in few-shot, and 53\% in zero-shot tasks without any hyper-parameter tuning. All methods are encapsulated as a plug-and-play module

Figures (12)

• Figure 1: Dynamical systems embody physical laws unfolding in space and time, with time series as the low-dimensional observations. Our embedding duality theory bridges these two frameworks, demonstrating that parameterized hidden state representations are the model's estimation of dynamical system structures.
• Figure 2: Performance comparison of physics-guided time series embedding versus original method across eight aspects.
• Figure 3: Existing parameterized embedding (a-c) and physics-guided non-parametric embedding (d-f) techniques. (a) Each time series patch utilizes a shared linear projection layer to obtain hidden representations. (b) Dense time series are processed using windowed spectral transformations with gradients for adaptability. (c) Multivariant time series are embedded using a shared linear layer on a grid measure. (d) Time Delay embedding based on predetermined hyper-parameters. (e) Higher-order derivative values are concatenated to reconstruct dynamical structures. (f) Integral terms can replace higher-order derivatives to address numerical instability.
• Figure 4: Illustration for (a) Parameterized embedding and (b) Physics-guided embedding technology.
• Figure 5: Embedding Visualizations.

Theorems & Definitions (15)

• Definition 1: Dynamical System
• Proposition 4.1
• Proposition 4.2
• Lemma 4.3
• Lemma 4.4
• Conjecture 4.5: Dim Scaling Law
• Conjecture 4.6: Spurious Dynamics
• Proposition B.1
• proof
• Proposition B.2