Kinematic Tokenization: Optimization-Based Continuous-Time Tokens for Learnable Decision Policies in Noisy Time Series

TL;DR

This work tackles learning under noisy continuous-time signals by introducing Physics-Informed Tokenization with optimization-based spline enrichment to produce explicit, high-order kinematic tokens for Transformers. By extracting tokens that encode position, velocity, acceleration, and jerk (and corresponding volume-derived measures), the approach yields a continuous-time representation that remains learnable under an asymmetric risk objective and abstention. In a backtest across six US equities, SplineGPT avoids the Liquidation Equilibrium that plagues discrete baselines, delivering calibrated, non-trivial trading policies with improved downside protection and regime-adaptive behavior. The method demonstrates robust performance under varying transaction costs and volatility thresholds, suggesting broad applicability to domains where continuous dynamics are observed through noisy samples and where abstention can be rational. Overall, explicit continuous-time tokens grounded in physics offer a scalable path to more stable, interpretable AI policies in real-world, noisy time series.

Abstract

Transformers are designed for discrete tokens, yet many real-world signals are continuous processes observed through noisy sampling. Discrete tokenizations (raw values, patches, finite differences) can be brittle in low signal-to-noise regimes, especially when downstream objectives impose asymmetric penalties that rationally encourage abstention. We introduce Kinematic Tokenization, an optimization-based continuous-time representation that reconstructs an explicit spline from noisy measurements and tokenizes local spline coefficients (position, velocity, acceleration, jerk). This is applied to financial time series data in the form of asset prices in conjunction with trading volume profiles. Across a multi-asset daily-equity testbed, we use a risk-averse asymmetric classification objective as a stress test for learnability. Under this objective, several discrete baselines collapse to an absorbing cash policy (the Liquidation Equilibrium), whereas the continuous spline tokens sustain calibrated, non-trivial action distributions and stable policies. These results suggest that explicit continuous-time tokens can improve the learnability and calibration of selective decision policies in noisy time series under abstention-inducing losses.

Figures (7)

• Figure 1: Conceptual Framework for Physics-Informed Continuous Tokenization. The approach integrates models of system dynamics and measurement protocols, both subject to stochastic uncertainty. Unlike traditional filtering (e.g. Kalman) which yields discrete estimates via implicit representations, our optimization-based approach produces an explicit continuous spline representation. This continuum is necessary to extract high-order kinematic tokens for the foundation model.
• Figure 2: Continuous Tokenization Pipeline. The raw discrete data is transformed and then enriched using optimization-based stochastic spline generation Kearney2024. The coefficients representing Position, Velocity, Acceleration, and Jerk are extracted, anchored to the window start, and fed into a Causal Transformer.
• Figure 3: Behavioral Divergence and Calibration (NVDA).(A) Action distribution shows the discrete baseline (PatchTST, Red) collapsing to a narrow density peak (Liquidation Equilibrium), whereas SplineGPT (Blue) maintains a broad, active policy distribution. (B) Confusion matrix confirms SplineGPT achieves meaningful separation between Hold (2) and Action (0/1) states, rather than defaulting to a single class. (C) Calibration plot indicates the model's probability estimates for "Buy" signals track the empirical win rate, staying close to the diagonal (perfect calibration).
• Figure 4: Parameter Sensitivity Analysis (NVDA).(A)SplineGPT exhibits Rational Selectivity: as the volatility threshold ($\tau$) increases (making the task harder), the model monotonically reduces its action rate, avoiding false positives. (B) Performance remains robust (Sharpe $> 2.0$) even under high-friction regimes (up to 30bps), validating that the alpha generated by the kinematic tokens is not merely microstructure noise arbitrage.
• Figure 5: Kinematic Responsiveness (XOM). Rather than a failure, this false breakout illustrates the model's rapid adaptation. Although high velocity ($c_1$) triggered an initial entry, the immediate rollover in acceleration ($c_2$, Panel C) alerted the model to the failed breakout. The policy flipped to "Sell" (Red Triangles) within days, protecting capital from the subsequent $\approx 10\%$ decline. This rapid reaction speed explains the low Max Drawdown metrics relative to the market.