Optimal Uncertainty-guided Neural Network Training

TL;DR

This paper addresses unreliable convergence and inconsistent PI quality in NN-based direct PI construction. It proposes a smooth, customizable loss called $CWFDC$ that minimizes PINAW, PINAFD, and a squared coverage penalty $(1-α+δ - PICP)^2$, balancing PI width, coverage, and distance from targets. The method is evaluated on wind power and electricity demand data, showing up to 99.8% training convergence and reduced PI variation compared with prior methods like LUBE, Wan, Marin, and Zhang. The approach enables user-driven trade-offs among PI sharpness, coverage, and robustness to data shifts, with practical benefits for power-grid decision-making.

Abstract

The neural network (NN)-based direct uncertainty quantification (UQ) methods have achieved the state of the art performance since the first inauguration, known as the lower-upper-bound estimation (LUBE) method. However, currently-available cost functions for uncertainty guided NN training are not always converging and all converged NNs are not generating optimized prediction intervals (PIs). Moreover, several groups have proposed different quality criteria for PIs. These raise a question about their relative effectiveness. Most of the existing cost functions of uncertainty guided NN training are not customizable and the convergence of training is uncertain. Therefore, in this paper, we propose a highly customizable smooth cost function for developing NNs to construct optimal PIs. The optimized average width of PIs, PI-failure distances and the PI coverage probability (PICP) are computed for the test dataset. The performance of the proposed method is examined for the wind power generation and the electricity demand data. Results show that the proposed method reduces variation in the quality of PIs, accelerates the training, and improves convergence probability from 99.2% to 99.8%.

Figures (8)

• Figure 1: Importance of the uncertainty quantification. The point prediction, presented by the red line with a constant error possibility (the root mean square error) cannot represent the heteroscedastic uncertainty. PIs, represented by green lines becomes narrow in the less uncertain regions and become wide in more uncertain regions. Therefore, PIs can represent heteroscedastic uncertainty.
• Figure 2: The advantage of the NN-based direct PI construction over traditional and conditional PI construction techniques for a non-Gaussian probability distribution. The NN-optimization technique finds an optimal PI for any arbitrary probability distribution.
• Figure 3: A rough diagram presenting NN based PIs when the optimization considers only PICP and PINAW. PIs cover 80% to 90% targets but fail to predict sharp changes. Successful and unsuccessful PIs are represented by green and red lines respectively with upper and lower bound marks.
• Figure 4: The structure of the NN with input-output combinations. Four recent samples and the time is applied to quantify the uncertainty on the next sample.
• Figure 5: NN-size optimization for the uncertainty quantification of the wind power generation of the UK grid. (a) The LUBE cost function. Optimized NN-sizes are 9 for $\alpha$=5%, 8 for $\alpha$=10% and 7 for $\alpha$=20%. (b) C. Wan's cost function. Optimized NN-sizes are 10 for $\alpha$=5%, 9 for $\alpha$=10% and 9 for $\alpha$=20%. (c) L. G. Marn's cost function. Optimized NN-sizes are 11 for $\alpha$=5%, 10 for $\alpha$=10% and 10 for $\alpha$=20%. (d) G Zhang's cost function. Optimized NN-sizes are 10 for $\alpha$=5%, 9 for $\alpha$=10% and 9 for $\alpha$=20%. (e) the proposed cost function. Optimized NN-size is 10 for $\alpha$=5%, 10%, and 20%.