Bias- and Variance-Aware Probabilistic Rounding Error Analysis for Floating-Point Arithmetic

TL;DR

This work revisits probabilistic rounding error analysis in a moment-aware setting by derive a confidence-calibrated reformulation of the Higham and Mary bound that makes its confidence parameter explicit and introduces a variance-informed probabilistic backward error bound based on the first two moments of $\log(1+\delta)$.

Abstract

Probabilistic rounding error analysis can yield much sharper bounds than classical worst-case theory, but existing results typically rely on zero-mean rounding errors and often leave the confidence parameter implicit. This work revisits probabilistic rounding error analysis in a moment-aware setting. We first derive a confidence-calibrated reformulation of the Higham and Mary [16] bound that makes its confidence parameter explicit. We then introduce a variance-informed probabilistic backward error bound based on the first two moments of $\log(1+δ)$, where $δ$ is the relative rounding error. This allows the analysis to accommodate biased rounding error models rather than relying on a zero-mean assumption. To illustrate this framework, we study both a uniform model and a log-space $\operatorname{Beta}$ model for rounding errors, the latter of which provides a simple way to represent bias. This perspective shows that the growth of probabilistic rounding error bounds is not universal: near-zero-mean regimes recover $\sqrt{n}$-like behavior, while biased models can exhibit faster accumulation. $\texttt{CUDA}$ experiments in single and half precision on dot products, sparse matrix-vector products, and a stochastic boundary-value problem show that the proposed framework is especially useful in low-precision regimes where deterministic bounds are overly conservative and where bias-aware modeling better matches observed error growth.

Figures (11)

• Figure 1: Empirical distribution (left) and conditional expectation (right) of rounding error random variable $\delta = \frac{\mathrm{fl}( S_{n} + X_{n+1} ) - (S_n+X_{n+1})}{(S_n+X_{n+1})}$, where $S_n = \sum_{i=1}^n X_i$ and $X_i~\sim\mathcal{U}(0, 1)$ for all $i$. Here, computations are performed using half-precision floating-point arithmetic. To obtain the statistics, $10^4$ independent experiments were conducted for each $n$.
• Figure 1: Backward error and its bounds for the dot product of random vectors of size $n$ distributed as $\mathcal{U}(0,1)$ (left) and $\mathcal{U}(-1,1)$ (right), computed using single-precision arithmetic. All probabilistic bounds are evaluated using a confidence level $\mathcal{Q}(n;\zeta) = 0.99$, and the $\beta$-model uses the shape parameter $\beta = 2.0$. To obtain the statistics, $10^2$ independent experiments were conducted for each $n$. All bounds are plotted until they exceed one (), beyond which they are not meaningful for backward error analysis.
• Figure 2: An illustration of the random walk $\sum_{i=1}^{n}\log(1+\delta_i)$, where $\delta = \frac{\mathrm{fl}( S_{n} + X_{n+1} ) - (S_n+X_{n+1})}{(S_n+X_{n+1})}$ with $S_n = \sum_{i=1}^n X_i$ and $X_i~\sim\mathcal{U}(0, 1)$ for all $i$. Here, computations are performed using half-precision floating-point arithmetic. To obtain the statistics, $10^4$ independent trajectories were computed, with solid lines denoting the sample mean and shaded regions denoting two standard deviations about the mean. For the $\beta$-model (\ref{['def:beta_rounding_error_model']}), we used shape parameters $\alpha = 1.5$ and $\beta = 2.0$.
• Figure 2: Backward error and its bounds for the dot product of random vectors of size $n$ distributed as $\mathcal{U}(0,1)$ (left) and $\mathcal{U}(-1,1)$ (right), computed using half-precision arithmetic. All probabilistic bounds are evaluated using a confidence level $\mathcal{Q}(n;\zeta) = 0.99$, and the $\beta$-model uses the shape parameter $\beta = 2.0$. To obtain the statistics, $10^2$ independent experiments were conducted for each $n$. All bounds are plotted until they exceed one (), beyond which they are not meaningful for backward error analysis.
• Figure 3: Comparison of rounding error bounds obtained using $\text{drea}$ (), $\text{mprea}$ (), and $\text{vprea}$ under the $\mathcal{U}$-model () and the $\beta$-model (,,). Results are shown for single-precision (left) and half-precision (right) floating-point arithmetic. All probabilistic bounds are evaluated using a confidence level $\zeta = 0.99$, and the $\beta$-model uses the shape parameter $\beta = 2.0$. We choose the shape parameter $\alpha$ such that $\mathbb{E}[\delta]$ is strictly negative, thus accounting for the negative bias in rounding errors observed when adding small increments to a large sum, as shown in \ref{['fig:rounding_error_distribution_small_increments']}. Here, we plot $\gamma_n$, $\tilde{\gamma}_n$, and $\hat{\gamma}_n$ until they exceed one (), beyond which they are not meaningful for backward error analysis.

Theorems & Definitions (27)

• Lemma 2.1: Deterministic rounding error analysis
• Lemma 2.2: Hoeffding's Inequality
• Corollary 3.1: Mean-informed probabilistic rounding error analysis
• Proof 1
• Lemma 3.2: Bernstein's Inequality
• Theorem 3.3: Variance-informed probabilistic rounding error analysis
• Proof 2
• Definition 3.4: $\mathcal{U}$-model
• Definition 3.5: $\beta$-model
• Proposition 3.6