Measurement Uncertainty: Relating the uncertainties of physical and virtual measurements
Simon Cramer, Tobias Müller, Robert H. Schmitt
TL;DR
This work formalizes a principled link between physical measurement uncertainty, as defined by the GUM, and the predictive uncertainty of probabilistic machine learning models used for virtual measurements in manufacturing. It defines the virtual measurement quantities hat $y(\mathbf{x})$ and hat $\sigma(\mathbf{X})$, and derives their uncertainty decomposition as $\hat{\sigma}^2(\mathbf{X}) = \mathbb{E}_{p(\mathbf{w}|\mathcal{D})}[\sigma_n^2(\mathbf{X};\mathbf{w})] + \mathbb{V}_{p(\mathbf{w}|\mathcal{D})}[f(\mathbf{X};\mathbf{w})]$, with hat $y(\mathbf{x}) = \mathbb{E}_{p(\mathbf{w}|\mathcal{D})}[f(\mathbf{x};\mathbf{w})]$. The paper proposes a three-stage process for both physical and virtual measurements, employing Bayesian inference and variational inference to train probabilistic virtual models and discusses documentation practices to ensure comparability with physical measurements. By establishing equivalence in uncertainty treatment, the authors enable the use of virtual measurements within established quality-management frameworks, potentially achieving 100% inspection while enabling risk quantification in predictive quality. This approach promises substantial value for complex process chains by prompt fault detection and the possibility to abort downstream steps when uncertainty is high.
Abstract
In the context of industrially mass-manufactured products, quality management is based on physically inspecting a small sample from a large batch and reasoning about the batch's quality conformance. When complementing physical inspections with predictions from machine learning models, it is crucial that the uncertainty of the prediction is known. Otherwise, the application of established quality management concepts is not legitimate. Deterministic (machine learning) models lack quantification of their predictive uncertainty and are therefore unsuitable. Probabilistic (machine learning) models provide a predictive uncertainty along with the prediction. However, a concise relationship is missing between the measurement uncertainty of physical inspections and the predictive uncertainty of probabilistic models in their application in quality management. Here, we show how the predictive uncertainty of probabilistic (machine learning) models is related to the measurement uncertainty of physical inspections. This enables the use of probabilistic models for virtual inspections and integrates them into existing quality management concepts. Thus, we can provide a virtual measurement for any quality characteristic based on the process data and achieve a 100 percent inspection rate. In the field of Predictive Quality, the virtual measurement is of great interest. Based on our results, physical inspections with a low sampling rate can be accompanied by virtual measurements that allow an inspection rate of 100 percent. We add substantial value, especially to complex process chains, as faulty products/parts are identified promptly and upcoming process steps can be aborted.