Uncertainty in Machine Learning
Hans Weytjens, Wouter Verbeke
TL;DR
Uncertainty in Machine Learning surveys the distinction between epistemic and aleatoric uncertainty and presents practical methods to quantify and manage uncertainty across linear regression, random forests, and neural networks. It introduces conformal prediction as a model-agnostic framework for producing prediction sets and intervals with finite-sample guarantees, and demonstrates how to use entropy-based measures and accuracy-rejection curves to calibrate and compare uncertainty estimates. The chapter also discusses Bayesian neural networks and dropout-based approximations as scalable means to capture both epistemic and aleatoric uncertainty in regression and classification, with an explicit loss formulation for aleatoric noise. The practical impact is to enable risk-aware decision making, automated triage, and earlier deployment of prediction systems by leveraging quantified uncertainty and calibration tools.
Abstract
This book chapter introduces the principles and practical applications of uncertainty quantification in machine learning. It explains how to identify and distinguish between different types of uncertainty and presents methods for quantifying uncertainty in predictive models, including linear regression, random forests, and neural networks. The chapter also covers conformal prediction as a framework for generating predictions with predefined confidence intervals. Finally, it explores how uncertainty estimation can be leveraged to improve business decision-making, enhance model reliability, and support risk-aware strategies.