Uncertainty separation via ensemble quantile regression

TL;DR

The paper tackles reliable uncertainty quantification by separating aleatoric and epistemic uncertainty in data-driven modeling. It introduces Ensemble Quantile Regression (E-QR) and a progressive sampling algorithm that iteratively enriches data in regions of high uncertainty to stabilize the type separation. Compared with Deep Ensembles and Monte Carlo dropout, E-QR yields improved aleatoric estimates while preserving epistemic insights, and its workflow is scalable to large datasets. Experiments on synthetic toy problems and a multi-joint robotic arm demonstrate robust separation with reduced leakage and accurate uncertainty localization.

Abstract

This paper introduces a novel and scalable framework for uncertainty estimation and separation with applications in data driven modeling in science and engineering tasks where reliable uncertainty quantification is critical. Leveraging an ensemble of quantile regression (E-QR) models, our approach enhances aleatoric uncertainty estimation while preserving the quality of epistemic uncertainty, surpassing competing methods, such as Deep Ensembles (DE) and Monte Carlo (MC) dropout. To address challenges in separating uncertainty types, we propose an algorithm that iteratively improves separation through progressive sampling in regions of high uncertainty. Our framework is scalable to large datasets and demonstrates superior performance on synthetic benchmarks, offering a robust tool for uncertainty quantification in data-driven applications.

Figures (4)

• Figure 1: On the left figure, we observe how the lack of data in a region with aleatoric uncertainty causes each sub-network to fit the noise differently, as they only access small subsets of the data. This overfitting results in the false reporting of epistemic uncertainty where none exists. On the right figure, we see that in regions lacking sufficient data, the fits for aleatoric uncertainty become unreliable, leading to incorrect report of aleatoric uncertainty.
• Figure 2: Multi joint robotic arm with a moving base and 4 rotating joints. The goal is to train a model that can predict the 2D position of the tip of the arm.
• Figure 3: In the top figure, from left to right, we present the original training data, the model’s predictions, and the aleatoric and epistemic uncertainty maps. The first row corresponds to the first output, while the second row corresponds to the second output of the model. The epistemic uncertainty map highlights four regions: two caused by a lack of data and two influenced by the leak of random noise. To achieve accurate separation, we apply Algorithm \ref{['alg:u_separation']}. After two iterations, only the regions with aleatoric uncertainty remain in the uncertainty map, confirming that the vanished uncertain areas were indeed epistemic. Note that the white dots are due to the low density of the training samples.
• Figure 4: This experiment highlights the leakage of epistemic uncertainty into aleatoric uncertainty. The plot illustrates the cross-section of two out of four rotating joints and their effect on the 2D position of the tip. From left to right, we present the original training data, the model’s predictions, and the aleatoric and epistemic uncertainty maps. Since no random noise is injected into this problem, we expect to observe only epistemic uncertainty. However, the aleatoric uncertainty map incorrectly reflects leaked epistemic uncertainty. Applying Algorithm \ref{['alg:u_separation']} for four iterations resolves this issue, as the uncertainty vanishes when the uncertain regions are filled with additional data, confirming that the observed uncertainty was indeed epistemic.