Uncertainty calibration for probabilistic projection methods
Vladimir Fanaskov
TL;DR
This work addresses the mismatch between probabilistic projections and actual error in linear solves by augmenting the prior covariance with a subspace spanning ${\sf Null}(W^{T}A)$, yielding nontrivial posterior uncertainty while preserving the projection mean. It develops practical, computable covariance constructions using projector-based forms for $\boldsymbol{ abla}_0 = VV^{T} + Y G Y^{T}$, analyzes when probabilistic projection is sound, and provides a Krylov-specific uncertainty calibration procedure that relies on an extra calibration run to align posterior statistics with observed errors. The authors demonstrate that Reid's covariance is a special case of their general framework, and they show through extensive numerical experiments that their calibration approach substantially improves uncertainty quality over existing methods, albeit with higher calibration cost. They also illustrate a PDE-constrained optimization example to highlight practical implications and potential use in surrogate modeling and optimization. Overall, the paper advances probabilistic linear solves by enabling calibrated, interpretable uncertainty in projection methods and clarifies the limitations of applying these ideas directly to Krylov subspace methods.
Abstract
Classical Krylov subspace projection methods for the solution of linear problem $Ax = b$ output an approximate solution $\widetilde{x}\simeq x$. Recently, it has been recognized that projection methods can be understood from a statistical perspective. These probabilistic projection methods return a distribution $p(\widetilde{x})$ in place of a point estimate $\widetilde{x}$. The resulting uncertainty, codified as a distribution, can, in theory, be meaningfully combined with other uncertainties, can be propagated through computational pipelines, and can be used in the framework of probabilistic decision theory. The problem we address is that the current probabilistic projection methods lead to the poorly calibrated posterior distribution. We improve the covariance matrix from previous works in a way that it does not contain such undesirable objects as $A^{-1}$ or $A^{-1}A^{-T}$, results in nontrivial uncertainty, and reproduces an arbitrary projection method as a mean of the posterior distribution. We also propose a variant that is numerically inexpensive in the case the uncertainty is calibrated a priori. Since it usually is not, we put forward a practical way to calibrate uncertainty that performs reasonably well, albeit at the expense of roughly doubling the numerical cost of the underlying projection method.