Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

TL;DR

This work tackles the problem of reliable uncertainty quantification in regression by learning prediction intervals (PIs) alongside point targets using two companion neural networks. It introduces DualAQD, a dual-objective loss that minimizes PI width while enforcing PI integrity through a differentiable penalty, balanced by a self-adaptive coefficient, batch-sorting, and MC-Dropout-based model uncertainty estimation. Across synthetic data, eight benchmark datasets, and a real crop-yield regression task, DualAQD delivers significantly narrower PIs while maintaining nominal coverage ($PICP \ge 1-\alpha$, with $\alpha=0.05$) and comparable target accuracy, outperforming state-of-the-art NN-based methods such as QD-Ens, QD+, and MC-Dropout-PI. The approach reduces hyperparameter tuning, relies on two specialized networks rather than ensembles, and demonstrates strong adaptability to varying uncertainty levels, with practical implications for reliable regression in real-world applications.

Abstract

Accurate uncertainty quantification is necessary to enhance the reliability of deep learning models in real-world applications. In the case of regression tasks, prediction intervals (PIs) should be provided along with the deterministic predictions of deep learning models. Such PIs are useful or "high-quality" as long as they are sufficiently narrow and capture most of the probability density. In this paper, we present a method to learn prediction intervals for regression-based neural networks automatically in addition to the conventional target predictions. In particular, we train two companion neural networks: one that uses one output, the target estimate, and another that uses two outputs, the upper and lower bounds of the corresponding PI. Our main contribution is the design of a novel loss function for the PI-generation network that takes into account the output of the target-estimation network and has two optimization objectives: minimizing the mean prediction interval width and ensuring the PI integrity using constraints that maximize the prediction interval probability coverage implicitly. Furthermore, we introduce a self-adaptive coefficient that balances both objectives within the loss function, which alleviates the task of fine-tuning. Experiments using a synthetic dataset, eight benchmark datasets, and a real-world crop yield prediction dataset showed that our method was able to maintain a nominal probability coverage and produce significantly narrower PIs without detriment to its target estimation accuracy when compared to those PIs generated by three state-of-the-art neural-network-based methods. In other words, our method was shown to produce higher-quality PIs.

Figures (7)

• Figure 1: An example of our PI-generation method on a synthetic dataset.
• Figure 2: $\mathcal{L}_3$ penalty calculation, (a) without batch sorting; (b) with batch sorting.
• Figure 3: Performance of PI generation methods on the synthetic dataset.
• Figure 4: Box plots of the $MPIW_{val}$ and $MSE_{val}$ scores of DualAQD, QD+, QD-Ens, and MC-Dropout-PI PI generation methods on the synthetic and benchmarking datasets: (a) Synthetic. (b) Boston. (c) Concrete. (d) Energy. (e) Kin8nm. (f) Power. (g) Protein. (h) Yacht. (i) Year.
• Figure 5: $MPIW$ and $PICP$ learning curves obtained for the Power dataset using DualAQD. (a)$\eta=0.01$. (b)$\eta=0.1$.