HopCast: Calibration of Autoregressive Dynamics Models

TL;DR

HopCast reframes autoregressive dynamics forecasting by introducing a Predictor-Corrector framework that uses Modern Hopfield Networks to retrieve context-dependent errors. This avoids conventional uncertainty propagation and yields calibrated, sharp multi-step prediction intervals across diverse dynamical systems. The approach demonstrates strong calibration (CE) and competitive predictive accuracy (MSE), with a tunable Attention Span via the sequence length $S_L$ to control interval width. It also showcases model-based reinforcement learning viability, where HopCast serves as an uncertainty-aware dynamics module with competitive task performance. Limitations include timestep-context reliance and horizon generalization, suggesting future work on discretization-invariant, scalable variants.

Abstract

Deep learning models are often trained to approximate dynamical systems that can be modeled using differential equations. Many of these models are optimized to predict one step ahead; such approaches produce calibrated one-step predictions if the predictive model can quantify uncertainty, such as Deep Ensembles. At inference time, multi-step predictions are generated via autoregression, which needs a sound uncertainty propagation method to produce calibrated multi-step predictions. This work introduces an alternative Predictor-Corrector approach named \hop{} that uses Modern Hopfield Networks (MHN) to learn the errors of a deterministic Predictor that approximates the dynamical system. The Corrector predicts a set of errors for the Predictor's output based on a context state at any timestep during autoregression. The set of errors creates sharper and well-calibrated prediction intervals with higher predictive accuracy compared to baselines without uncertainty propagation. The calibration and prediction performances are evaluated across a set of dynamical systems. This work is also the first to benchmark existing uncertainty propagation methods based on calibration errors.

Figures (9)

• Figure 1: Predictor-Corrector mechanism
• Figure 2: Training and Inference with Correction Model ($\text{Encoder}_x+\text{MHN}_x$).
• Figure 3: An example set of errors ($E_x$) sampled from Association Memory at Timestep 106 during autoregression for the x-output of the Predictor for the Lorenz system, where the red dotted line denotes the ground truth error.
• Figure 4: (a) Upper center: Sine function with heteroscedastic noise (b) Lower left: Attention Span for $x = 3.28$ as a query with $\mathtt{S_L}=3$ (c) Lower right: Attention Span for $x = 3.28$ as a query with $\mathtt{S_L}=8$
• Figure 5: (a). Upper left: HopCast (b). Upper right: Expectation (c). Lower left: Moment Matching (d). Lower right: Trajectory Sampling. A comparison of prediction intervals generated by HopCast and three uncertainty propagation approaches. The x-axis denotes the time steps, and the y-axis shows the $x$ output of the Lorenz system. The Upper PI and Lower PI show the $m=9$ equally spaced prediction intervals from 10% to 90%.