FlowMixer: A Constrained Neural Architecture for Interpretable Spatiotemporal Forecasting

TL;DR

FlowMixer introduces a constrained neural architecture for interpretable spatiotemporal forecasting by embedding nonnegative matrix mixing within a reversible mapping. The core transform $F(X,W_t,W_f,φ)=φ^{-1}(W_t φ(X) W_f^T)$ supports a Kronecker-Koopman eigenmode decomposition, enabling interpretable space–time patterns and direct algebraic horizon modification. A Semi-Orthogonal Basic Reservoir (SOBR) and Time-Dependent RevIN further bolster stability and chaotic dynamics modeling, yielding robust long-horizon performance. Across time-series benchmarks, chaotic attractors, and 2D turbulence simulations, FlowMixer achieves competitive accuracy while enhancing interpretability, demonstrating that architectural constraints can strengthen both predictive power and scientific insight.

Abstract

We introduce FlowMixer, a neural architecture that leverages constrained matrix operations to model structured spatiotemporal patterns. At its core, FlowMixer incorporates non-negative matrix mixing layers within a reversible mapping framework-applying transforms before mixing and their inverses afterward. This shape-preserving design enables a Kronecker-Koopman eigenmode framework that bridges statistical learning with dynamical systems theory, providing interpretable spatiotemporal patterns and facilitating direct algebraic manipulation of prediction horizons without retraining. Extensive experiments across diverse domains demonstrate FlowMixer's robust long-horizon forecasting capabilities while effectively modeling physical phenomena such as chaotic attractors and turbulent flows. These results suggest that architectural constraints can simultaneously enhance predictive performance and mathematical interpretability in neural forecasting systems.

Figures (17)

• Figure 1: Overview of FlowMixer's constrained architecture. The core architecture (bottom) processes input sequences using three key components: (1) a reversible mapping, mainly reversible instance normalization (RevIN) to handle distribution shifts, (2) a constrained mixing layer with positive time mixing ($W_t$) and stochastic feature mixing ($W_f$) matrices, and (3) adaptive skip connections embedded within the mixing matrices. The spatiotemporal evolution (top) demonstrates how data is padded to create square mixing matrices, enabling eigendecomposition and efficient temporal modeling and interpretation. The Kronecker-Koopman eigenmodes (right), derived from the architecture's mathematical structure ($W_f \otimes W_t = PDP^{-1}$), provide a space-time decomposition of the mixer input as a weighted sum of space-time eigenmodes.
• Figure 2: Visualization of Kronecker-Koopman Eigenmodes for traffic dataset. (a) Time eigenvalues distribution, with aligned angular values indicating the periodic nature of the traffic datasets. (b-d) 3 first time eigenvectors (real part). (e,i) First and Third space eigenvectors, with the first showing constant values consistent with the stochastic space mixing matrix (representing a Markov transition process). (f-h, j-l) Real parts of Kronecker-Koopman Eigenmodes revealing space-time patterns. The periodic structures in time eigenvectors and coherent patterns in higher-mode products demonstrate how FlowMixer captures spatiotemporal dynamics.
• Figure 3: Predictions of Lorenz (a), Rössler (b), and Aizawa (c) chaotic attractors (scaled [-1,1]). Each row shows the evolution of x, y, and z variables over time. Ground truth (black), FlowMixer (blue), Reservoir Computing (orange), and N-BEATS (purple) trajectories are compared. While all methods capture the Lorenz attractor's structure, N-BEATS shows a notable deviation in the Rössler system, and Reservoir Computing shows differences particularly in the z-component predictions. FlowMixer exhibits consistent performance across all three systems. Experimental settings are summerazied in Appendix \ref{['app:hyper']} and presented in detail in Appendix \ref{['app:extra_hyper_chaos']}.
• Figure 4: Prediction of vorticity fields for flow past a cylinder at $Re=150$ (left columns) and a NACA airfoil at $Re=1000$ (right columns). (a) Ground truth vorticity fields showing the evolution of wake structures. (b) FlowMixer predictions demonstrate accurate capture of both near-body structures and downstream vortex evolution. (c) Error fields with corresponding Mean Squared Error (MSE) values show minimal discrepancy between prediction and ground truth across varying geometries and flow regimes. Experimental details are provided in Appendix \ref{['app:2dturb']}. Additional flow visualizations and analyses are provided in Appendix \ref{['app:convLSTM']} and Appendix \ref{['app:SGDvsADAM']}.
• Figure 5: FFT Spectral Analysis of Periodicity Patterns in Benchmark Time Series Datasets. The plots show magnitude versus period (in hours) on logarithmic scales for eight standard forecasting datasets. Significant periodicities are marked with red dots and vertical lines, revealing prominent daily (24h) patterns across all datasets, with additional weekly (168h) and biweekly cycles of varying strengths. These spectral characteristics inform our model design, particularly for incorporating appropriate time mixing components and periodicity-aware architectures to capture multi-scale temporal dependencies. We select mainly periods of [24,168] in our experiments.