A Phenomenological AI Foundation Model for Physical Signals

TL;DR

This work develops and trained a model that could effectively encode and predict physical behaviors, such as mechanical motion and thermodynamics, including phenomena not seen in training, and proposes a phenomenological approach and framework for creating and validating such AI foundation models.

Abstract

The objective of this work is to develop an AI foundation model for physical signals that can generalize across diverse phenomena, domains, applications, and sensing apparatuses. We propose a phenomenological approach and framework for creating and validating such AI foundation models. Based on this framework, we developed and trained a model on 0.59 billion samples of cross-modal sensor measurements, ranging from electrical current to fluid flow to optical sensors. Notably, no prior knowledge of physical laws or inductive biases were introduced into the model. Through several real-world experiments, we demonstrate that a single foundation model could effectively encode and predict physical behaviors, such as mechanical motion and thermodynamics, including phenomena not seen in training. The model also scales across physical processes of varying complexity, from tracking the trajectory of a simple spring-mass system to forecasting large electrical grid dynamics. This work highlights the potential of building a unified AI foundation model for diverse physical world processes.

Figures (8)

• Figure 1: The $\Omega$-Framework describes the relationship between quantities $q_i$, sensors $s_{j}$ and measurements $m_{i,j}$ in the physical world to inform a phenomenological AI model capable of describing past ($t \leq T$) and future ($t > T$) trajectories of any physical quantity of interest $q_i(t)$.
• Figure 2: Architecture: Sensor data is normalized and segmented into fixed-length, one-dimensional patches that are used to train a transformer-based encoder network. Compact downstream decoders are then employed to reconstruct and predict the trajectories of the physical quantities of interest.
• Figure 3: Harmonic oscillator experiment. Left: experimental set up consist of a spring-mass system. Right: Sample forecasts for three regions of oscillatory behavior: (a) semi-chaotic dampened oscillation, (b) transition to dampened harmonic oscillation, and (c) dampened harmonic oscillation.
• Figure 4: Thermodynamic experiment. Left: experimental set up; Right: example of (a) temperature of the two water baths, (b) the output electrical current induced by the temperature differential, (c) example forecasts over two windows labeled A and B, respectively.
• Figure 5: Sample graphs for complex real-world processes. a) and b) Reconstruction and forecasting of Turkey's power consumption; c) Forecasting the oil temperature of an electrical transformer; d) Forecasting water vapor concentration in Germany.