Multi-Horizon Time Series Forecasting of non-parametric CDFs with Deep Lattice Networks

TL;DR

This work addresses the challenge of non-parametric probabilistic forecasting for time series by forecasting an implicit CDF with monotonic constraints. It introduces a two-stage architecture where an LSTM embedding $f_1$ feeds a Deep Lattice Network $f_2$ that takes quantile inputs to produce a complete multi-horizon CDF while enforcing monotonicity to prevent quantile crossovers, enabling $h=36$ horizon forecasts from a $w=96$ past window. The authors compare this DLN-based SQR approach against QR, linear baselines, and scalable monotonic networks on day-ahead solar irradiance data, showing competitive accuracy and improved calibration across multiple metrics such as CRPS and ACE, with superior CDF coverage. The results demonstrate that monotonic networks can effectively model non-parametric CDFs in time series, offering practical benefits for risk-aware decision making in renewable energy and related domains, while highlighting trade-offs in model complexity and computation. The work lays a foundation for further cross-pollination between monotonic neural networks and probabilistic forecasting, encouraging exploration of alternative embeddings and larger lattice ensembles.

Abstract

Probabilistic forecasting is not only a way to add more information to a prediction of the future, but it also builds on weaknesses in point prediction. Sudden changes in a time series can still be captured by a cumulative distribution function (CDF), while a point prediction is likely to miss it entirely. The modeling of CDFs within forecasts has historically been limited to parametric approaches, but due to recent advances, this no longer has to be the case. We aim to advance the fields of probabilistic forecasting and monotonic networks by connecting them and propose an approach that permits the forecasting of implicit, complete, and nonparametric CDFs. For this purpose, we propose an adaptation to deep lattice networks (DLN) for monotonically constrained simultaneous/implicit quantile regression in time series forecasting. Quantile regression usually produces quantile crossovers, which need to be prevented to achieve a legitimate CDF. By leveraging long short term memory units (LSTM) as the embedding layer, and spreading quantile inputs to all sub-lattices of a DLN with an extended output size, we can produce a multi-horizon forecast of an implicit CDF due to the monotonic constraintability of DLNs that prevent quantile crossovers. We compare and evaluate our approach's performance to relevant state of the art within the context of a highly relevant application of time series forecasting: Day-ahead, hourly forecasts of solar irradiance observations. Our experiments show that the adaptation of a DLN performs just as well or even better than an unconstrained approach. Further comparison of the adapted DLN against a scalable monotonic neural network shows that our approach performs better. With this adaptation of DLNs, we intend to create more interest and crossover investigations in techniques of monotonic neural networks and probabilistic forecasting.

Figures (3)

• Figure 1: Architecture of the LSTM-DLN for SQR. If the DLN is looped over a set of quantiles, a correspondingly detailed approximation of a CDF can be accumulated.
• Figure 2: Two 36h forecasts of the test data from LSTM-DLN (row 1), LSTM-SMNN (row 2), and LSTM-NN (row 3). Shown are 11 quantiles between 0.025 and 0.975. Axis labels are consistent between subplots.
• Figure 3: PICP (left) and reliability diagram (right). PICP starts at approx. 0.2 as that is the smallest interval around the median. The values are means across all 5 models, with standard deviations displayed as colored areas around the curve.