Towards Reliable Time Series Forecasting under Future Uncertainty: Ambiguity and Novelty Rejection Mechanisms

TL;DR

The paper addresses the challenge of reliable time series forecasting under future uncertainty by introducing a dual rejection mechanism that abstains predictions when confidence is low or inputs are novel. Ambiguity rejection estimates forecast uncertainty from historical error variance and uses prediction-interval thresholds, while novelty rejection leverages Variational Autoencoders and Mahalanobis distance to detect out-of-distribution inputs, combining these signals for a total rejection decision. The authors provide theoretical bounds for ideal and random rejection strategies and demonstrate empirically that the dual mechanism improves MAE and MSE across multiple datasets (ETT, Weather, Exchange Rate) and forecast horizons, with Ablation studies confirming the complementary value of both components. Practically, this work offers a reliable, end-to-end approach to strengthen forecasting under distributional shifts and concept drift without requiring future ground truth. The framework enables better risk management in real-world deployment by balancing rejection rate and predictive accuracy. $\mathcal{R}_{\\lambda}(\\Gamma)$, $Var(y-\\hat{y})$, prediction intervals, $D_M(\\boldsymbol{x})$, and VAEs form the core mathematical basis for uncertainty and novelty assessment.$

Abstract

In real-world time series forecasting, uncertainty and lack of reliable evaluation pose significant challenges. Notably, forecasting errors often arise from underfitting in-distribution data and failing to handle out-of-distribution inputs. To enhance model reliability, we introduce a dual rejection mechanism combining ambiguity and novelty rejection. Ambiguity rejection, using prediction error variance, allows the model to abstain under low confidence, assessed through historical error variance analysis without future ground truth. Novelty rejection, employing Variational Autoencoders and Mahalanobis distance, detects deviations from training data. This dual approach improves forecasting reliability in dynamic environments by reducing errors and adapting to data changes, advancing reliability in complex scenarios.

Figures (2)

• Figure 1: Distribution of sample losses predicted by the model. The vertical axis represents the density of the distribution, while the horizontal axis shows the specific values of the losses. The upper part displays the loss distribution without rejection, and the lower part shows the loss distribution including the rejection.
• Figure 2: Flowchart of the model. Here, $D(x)$ represents the Mahalanobis distance, $\delta(\mathbf{X})$ is the indicator function. When a sample is fed into the model, it first measures the distance between the sample and the high-dimensional distribution learned by the model. If the distance is large, the model chooses to reject the prediction for that sample. If the distance is small, the sample is passed to the rejector, which evaluates whether the model can make an accurate prediction. If the model cannot make an accurate prediction, the sample is rejected.